Block's research lab recently introduced its new voice model, Dessa. This innovation has garnered attention for its ability to run locally on end devices and reproduce human speech with unexpected authenticity, including distinct expressions such as laughter, pauses, and even hints of crying. According to the developers, the Dessa model is trained exclusively on human speech, and its operation requires only a single NVIDIA GPU unit, underscoring its efficiency.
A key feature of the Dessa model is its full-duplex operation. This means it can simultaneously speak and listen, which significantly speeds up and naturalizes conversation—much like in human interaction. Unlike common voice models that often rely on connections with large language models for information retrieval, Dessa draws its knowledge directly from the conversational data it was trained on.
Developers point out that current voice models often sound unnatural, even disruptive, as they rely on a textual intermediate step for speech generation. Many of them use special voice markers or tags to embed emotions into the generated speech. However, the company claims that with the Dessa model, emotions are an integral part of its architecture, although the mechanism for their control is not detailed.
In their blog post, the team criticizes existing methods of interacting with voice AI. They point out that converting voice to text removes a vast amount of information that text alone cannot convey. They state that even the most powerful AI systems today cannot perceive user emotions, which leads to poorer user experiences and potentially security risks.
Planen Sie eine neue Website oder App?
Schreiben Sie mir, was Sie vorhaben — ich schaue es mir gerne an und berate Sie zum besten Vorgehen.
The researchers demonstrated Dessa's capabilities through several conversational samples. These examples included situations where the assistant exhibited various voices, cadences, and emotions. For instance, in one conversation, the AI sounded tired when discussing work fatigue. In another sample, it displayed an emotional tone, almost breaking in its voice with emotion when discussing comfort provided by someone. A further example featured an interaction where a team member played a game with the voice assistant.
Interestingly, the model is designed to run locally on common consumer hardware, such as a MacBook M4 Max or a more powerful computer. However, public access to the Dessa model is not currently available. Instead, the company has invited security researchers to test the model, indicating an effort to thoroughly examine potential scenarios, for example, where a voice agent might unknowingly or intentionally mislead users.
Although Block primarily operates in the financial sector, offering solutions for small businesses—for instance, the Square platform already uses voice AI for order processing—the development of its own on-device voice model opens the door to a wide range of new business and consumer applications. This technology has the potential to significantly reduce costs and expand interaction possibilities.