5-HT2C and 5-HT1A Receptor Modulation of Biphasic Behavioral Responses to DOM in Mice

Study Background and Research Question

Serotonergic hallucinogens, such as (-)-2,5-dimethoxy-4-methylamphetamine (DOM), are valuable tools in neuropsychopharmacology for dissecting the roles of serotonin (5-HT) receptor subtypes in perception, cognition, and mood. DOM, a phenylalkylamine hallucinogen, is known to induce an inverted U-shaped (biphasic) dose-response in both head twitch response (HTR) and locomotor activity in rodents. While the involvement of the 5-HT2A receptor in mediating these effects is well-established, the mechanisms by which other 5-HT receptor subtypes contribute—especially at higher doses—remain poorly understood.

This knowledge gap is particularly relevant for refining preclinical models of hallucinogen action, as HTR is a predictive behavioral readout for psychedelic activity. Clarifying the receptor-specific contributions is essential for both basic neuroscience and the development of safer, more targeted neuropsychiatric therapeutics.

Key Innovation from the Reference Study

The reference study by Zhu et al. (2024) provides a detailed pharmacological dissection of the roles played by 5-HT2A, 5-HT2C, and 5-HT1A receptors in DOM-induced behavioral responses. Notably, the work demonstrates that the biphasic dose-response curves for both HTR and locomotor activity are shaped not solely by 5-HT2A activation, but also by complex interactions with 5-HT2C and 5-HT1A receptor signaling. The study also reveals that Gαi/o-coupled signaling (inhibited using pertussis toxin, PTX) modulates these behavioral outcomes, providing a mechanistic link to intracellular signal transduction pathways.

Methods and Experimental Design Insights

The investigators utilized C57BL/6J mice to systematically evaluate the effects of DOM across a range of doses (0.615–10 mg/kg, i.p.) on HTR and locomotor activity. To dissect receptor subtype contributions, selective pharmacological antagonists and agonists were employed:

• 5-HT2A antagonist: M100907
• 5-HT2C antagonist: SB242084
• 5-HT1A antagonist: WAY100635
• 5-HT1A agonist: 8-OH-DPAT

In addition, the study utilized central administration (i.c.v.) of pertussis toxin (PTX) to inhibit Gαi/o-coupled receptor signaling. Behavioral endpoints included quantification of HTR and automated measures of locomotor activity following drug administration.

Protocol Parameters

• DOM administration: 0.615–10 mg/kg, intraperitoneally, to induce dose-dependent effects.
• 5-HT2A antagonist (M100907): 500 μg/kg, i.p.; full blockade of HTR at all DOM doses; 50 μg/kg, i.p., reduces locomotor hyperactivity at low DOM doses.
• 5-HT2C antagonist (SB242084): 0.3–1 mg/kg, i.p.; reduces HTR at 2.5 mg/kg DOM; enhances locomotor activity across DOM doses.
• 5-HT1A antagonist (WAY100635): 1 mg/kg, i.p.; increases both HTR and locomotor activity induced by DOM.
• 5-HT1A agonist (8-OH-DPAT): 1 mg/kg, i.p.; reduces both HTR and locomotor activity induced by DOM.
• Pertussis toxin (PTX): 0.25 μg/mouse, i.c.v.; enhances HTR and attenuates locomotor activity induced by DOM.

Core Findings and Why They Matter

The study's results clarify several key aspects of serotonin receptor pharmacology:

• 5-HT2A receptor blockade (via M100907) fully suppresses HTR at all DOM doses, confirming its essential role in this behavioral effect. However, it only attenuates locomotor hyperactivity at lower DOM doses, suggesting additional mechanisms at play at higher doses.
• 5-HT2C receptor antagonism (via SB242084) selectively reduces HTR at a middle DOM dose (2.5 mg/kg) but broadly enhances locomotor activity, indicating a balancing or inhibitory role for 5-HT2C signaling in these behaviors.
• 5-HT1A receptor signaling is shown to be modulatory: antagonism increases, and agonism decreases both HTR and locomotor activity, underscoring its regulatory influence on hallucinogen-induced behaviors.
• Gαi/o protein inhibition with pertussis toxin (an AB5-type protein exotoxin) enhances HTR and reduces DOM-induced locomotor activity, linking cAMP-dependent signaling pathways to observable behavioral changes.

These findings highlight the complexity of serotonergic modulation underlying hallucinogen-induced behaviors and suggest that both 5-HT2C and 5-HT1A receptors contribute to the biphasic dose-response, with distinct mechanistic roles for each. The evidence supports a model in which receptor cross-talk and intracellular signaling cascades—potentially involving cAMP and G-protein coupling—finely tune the behavioral outcomes of serotonergic drug exposure.

Comparison with Existing Internal Articles

Several internal resources expand on the utility and mechanisms of pertussis toxin in immune and neuropharmacological research. For example, "Pertussis Toxin: Mechanistic Insights and Advanced Research Horizons" details how this AB5-type protein exotoxin modulates intracellular signaling, particularly through cAMP pathways—a mechanistic theme echoed in the current reference study. Similarly, "Pertussis Toxin in Immune Modulation: Protocols & Troubleshooting" provides applied workflows for using pertussis toxin to dissect immune modulation, which parallels its use here in probing G-protein-coupled receptor signaling. While these internal sources focus primarily on immunological contexts, the present study demonstrates the cross-applicability of these tools for dissecting neurobehavioral pathways.

Limitations and Transferability

While the study offers robust evidence for receptor-specific modulation of DOM-induced behaviors, several limitations must be acknowledged:

• Behavioral models in mice may not fully capture the complexity of hallucinogen effects in humans.
• Pharmacological manipulations, such as systemic administration of antagonists/agonists, may have off-target effects.
• The use of pertussis toxin to inhibit Gαi/o signaling is powerful but may not distinguish between effects on different cell types or brain regions.

Nevertheless, the mechanistic insights are highly transferable to other preclinical studies aiming to dissect serotonergic and cAMP-dependent pathways, especially in the context of neuropsychopharmacology and immune response modulation in dendritic cells.

Research Support Resources

For researchers seeking to explore similar G-protein-coupled receptor mechanisms—or to dissect cAMP signaling in neuroimmune or behavioral studies—high-quality pertussis toxin is an established tool. The Pertussis toxin (SKU B7273) from APExBIO offers a well-characterized, high-purity AB5-type protein exotoxin suitable for precise inhibition of Gαi/o proteins in cellular and animal models. Its validated use in both immunology and behavioral neuroscience contexts, as reflected in the internal mechanistic review, makes it a reliable choice for studies requiring immune response modulation in dendritic cells or cAMP pathway interrogation. Researchers are advised to adhere to recommended storage and handling protocols to maintain toxin activity for reproducible results.