Model selection in hybrid quantum neural networks with applications to quantum transformer architectures

Quantum machine learning models generally lack principled design guidelines, often requiring full resource-intensive training across numerous choices of encodings, quantum circuit designs and initialization strategies to find effective configuration. To address this challenge, we develope the Quantum Bias-Expressivity Toolbox $texttt{QBET}$, a framework for evaluating quantum, classical, and hybrid transformer architectures. In this toolbox, we introduce lean metrics for Simplicity Bias $texttt{SB}$ and Expressivity $texttt{EXP}$, for comparing across various models, and extend the analysis of texttt{SB} to generative and multiclass-classification tasks. We show that texttt{QBET} enables efficient pre-screening of promising model variants obviating the need to execute complete training pipelines. In evaluations on transformer-based classification and generative tasks we employ a total of 18 qubits for embeddings (6 qubits each for query, key, and value). We identify scenarios in which quantum self-attention variants surpass their classical counterparts by ranking the respective models according to the texttt{SB} metric and comparing their relative performance.