Recent advances in artificial speech synthesis and machine learning equip AI-powered conversational agents, from voice assistants to social robots, with the ability to mimic human emotional expression during their interactions with users. One unexplored development is the ability to design machine-generated voices that induce varying levels of “shakiness” (i.e., trembling) in the agents' voices. In the current work, we examine how the trembling voice of a conversational AI impacts users' perceptions, affective experiences, and their subsequent behavior. Across three studies, we demonstrate that a trembling voice enhances the perceived psychological vulnerability of the agent, followed by a heightened sense of empathic concern, ultimately increasing people's willingness to donate in a prosocial charity context. We provide further evidence from a large-scale field experiment that conversational agents with a trembling voice lead to increased click-through rates and decreased costs-per-impression in an online charity advertising setting. These findings deepen our understanding of the nuanced impact of intentionally designed voices of conversational AI agents on humans and highlight the ethical and societal challenges that arise.