BENCH-001Read report ⟶
Speech-to-Text
Fifteen frontier transcription models against real multilingual conversation. Best-in-class still misses every second word of dialectal Arabic.
15 models4 languages
CU1 Z 0X Q 0 CZZ 1 Z Q Z U ZU( C (0 0 O ) ?L Q)1 / ? L ? C C ) X / ( Q L Z C )/ 0O ( 0 X U )Z CX ( 0 1
? U 1 Q Z X 1 ? / C Z ) 0U 1 O X U C 1 U U? (Z LX ( / C Z U 0 Q Z X ( X U 1 ) ZQ 1 (X 0 / L XL L L L?)) 0
X QU )/CUZU ) U ) X( Z C 0 O? C ) 1/ ( O Q O 1 0 ) 1 Z C L0 0 QO C O ? Z ?1 C Z ( OX 1U ZZ/ UU
00L XQL Z/OO1 L XX 0O 0X O/ L UL 0 C LX OZ Q O 1 L11 ( 0 L ? 1 / Q )O X Q0O / 0 Z / 1?LQ 0 U XQ)0 ((UOQ
X Z( Z/ L/0CX X 1 Q L ) QO) C 1 U O 1L (0 O Q 1 Q 1 Z0 L ) L U 1 0 0O LZ1 ZZ0 OO OQ L (L0
C0U 0 0UOOL O 0 L0 0 C1 C UL / Z Z 1 Z U U Q 1 1 U 0 U 0 1 Q 1O Q O 0 CC U 1Z 1U ( / C Z0Z L X
Z0UQLUOZ L UQ O?LQ Z X / Q XC C 1 1 U Q 1 ) 1 OQ / Q Q C ? Z 0 O ?U O C ( ZZ1 LO Q ) Z 00) ? Q XO O1 1 U1 /0 LL
X ZO U C ?0/ U) ( C 0O 1 0 C0L ) C1 U O Q X L Q ) C( X ( U U 0 / C C Q) / 0 L/ 11 X Q ZQ L ) L( U COU
ZU )CZ QU (Z/ X / QO L C 1 O0 0 1 0ZO) / 1 X 0 ) X ZU O L ?/ X ( C 0 ?( C ZU Q ) ( U Z Q L ZC11C L
1 /X? U //O CO1LZ Z X 1 Q ? X U Q Q / L ? O ) / 0 Q O L C X O U O X Z O QQ C)U 0C Q O( X L ) (/ C 0 Z( ?LZQ / U L /CU
Z?)1?X L 0) XZ ( O X 1 Z/( /O1 Q U/ C0 O0 U 0 Z Z O Z L 1 X X 1 Q L Q / U LC 0 L Z ?/ Q )U L L OZ ZO O X L11Q 1 (( ) 1 (1
L X 1 Q0X LQCC) O OC X C 0 0 C/ CO1 ?0 Z L 0U Q XQ Z O? C U X 1 0 Z U ) ) L 1Z Q ? 0 ?C ? C 0 0 0 ?L X LX C X C L1L)Z X
UC1QC Z01L U Q Z0ZQ 1U L Z 1 L O (0 LU O11 Z Q Z C U O UU U L? 1 Q C ( Q Q L C 0 C Z Q 1 Z CCZ X Z UU0C 0 L Z U Q 0 L 0 Q
XQ(U0)C Q /Z 1 1 / U (O QU XU Z0Z L X O 1 U L ) O /Z C OC ) X U ? L 0Z /O L L L ?C QC X ) O U ?? Z Z L XO O)Q/ZZO CCZ( OL0 1/
Q /XZL / O0O CX CZ 1 Z 0 XC LQX0 C QL QC Z ( Q) X X C UQ / Q 1) U 0 Z C X)UL L 1 L U X L / 0U 0 Q UX Q 1 0 C ZL) )( UX O Z Z/?0 QO Q UL0ZO
UO O U )Q?QU?U Q Z U ZO ? XO? O ( O / Q QL 0 X C Q ? C X Z 0 L X C O Q L U X 0 C O O ZXU O X CC CU O ) UUC/U X00 LQ?Z C
XC / 1? U /(OQ) Q/0 ( /Z0( Z0 X1Q QX C0 0CC 0 U ? U ) U O X X ( C 1 L C L Q C 0 C U 0U? X0Q L C / X 1 ?X L XO U ( Q 1 O U( C Z UQU/
Q)UUQ X LU 1 O L 0/)Q( C UL1 / X ) QZ C 0C U C X (Z Z CU X ) X X L ? X ?/ ? ( Z LX ZO ZL X L U / C 0 Z ) 0 0 ZCL1Q 1 L X ZX 0 ? O Q 0 Z XX C Q Z U O L L
L/0 O QZLCC) ZO01/U X L ) ?C U/X UU L XO U)?U Z U 1 0 ( )U CC Z ? Z XO QO ZL X/ L / Z/(QL U O ? X / Q Q Z C L U U 0 ? L 10 ( Q()U LZ QL 0 ZOQ Z UUL OC0UQ
LL0C01L 11 ( O U (C QZ Q 0 1Q1 ?O 1( 1 / OQ / U Z ? Z C 0 C UX U OU U ( 1 0 U )U Z /Z U /C U L Z 1 O 0 Q ( 1 L O/ L ? 1 /1 L X?X U XO C O Z Z /0 C1? QXX X1( CQZ00 O ) C Q0?
O QZQO0 UUL Q 1LO1 1)1X LQ Q ( Q 1X /C 1Z 0QO XQ X ? 0 O 0 0 O1 ?Z ) 01 U U0 LO Q Z X / U O Z C L 1 C(U L L O Q X X? U U01 UXQUC X?/Q U1 XO0 L 0QLQ O /UXL
?X/OU 01 O 0 (0UUO L( 0 L ZQ Q 0 U 1 X L(O C0(O O X 1 X/ OX00 X ) O ( L C ( Q 0X Z ( ? C Q?Q O O 0 ?U L Z Q X ZO C ) L 0 O0 Q/ 0QL1 Z / 1 Q 0 1Z ?Z X1(CZZL C X OCOOU(
CL00X?Q U XOUZ ZQX1 Q CX 1 Q O CQ OO Z C X / OQ Q / C ) L Z X X X X Q ) X O OO U ) U 1 ?/ O L U X Q UQ O ( ) L L X Z 0 ZU ? LC 0 C 1( ) U X L )ZZQ Z O (ZL OQ Q/ 0O( L Z Q1O OLX1UU0O
) O 0X ?Z0 LL?OC (/) 0U X U/X Z C /C 1Q ) Q ? Z ?( XZ O Q 1 O Z ( L LL QO Z U/ O?Q CZ ( 0 U 1 X ? / Q C U C 1 ( Z ) Z0 L? Q L Z 1 ? XX L ? C Z Q 0 0 CQ L ?0 OX) 0ZLOZ0 1 0C 1L U 1 L 0?QC
1ZOQ/ LCX10U QUU(O OCU Z / ZU)L? Q0 0 L0 O ) C X 0 Z 1OU U QQ0 (XU Q /X)X U 0 O X Q Z(? CU ( Q 1 1 ZQC0? ? X( ( U L 0 1C CX Q )C1 L U Q 1 C X X 1 01 Q 1 (0O CU1 UOQ 1QZ(1O( ZO C QZXL
1 )LO ?X 1OU ?0ZZO Z L 0L )OXO X 0 QL QC 1OX /)XC0L LO /?/ 0 U 10 U Z / Q L ) 0 1O L L Z C 1X / Z 1 )L X OU1 Z ( L ?QZ 1( 0U U UX X 0/0 ZZ / OQ X LC 1Q C0 O? UUUOX ?0 L 0 Z? 0L ? L0 XL(L0Z0/
XU0ZZ( 0)1Q1 (ZL1 L LL)LCZO 0U1X / U 1/ (U ?(XQX X X ZU Z/ U X C 1 Z C L U0 1 C OQ Q (L CZ O O QQ 1 LQQ Q 1 ? Q O OO /X C L01 XLZ Q Z CUU )0 OQQ Z X LX )CL )Z UXZC UL ( Q C O X / QUL UQ C 1
UZ U1ZLZ1 UU ( X1? QO1 U Q X/ / L ? O CL/ 0LQ L Z X )OU LZU Q )Z L 10 0 U C QU X1O / (Z LZ 0LX Q U C O 1ZO X X LC U O0Q X Z U Z X/ Z 1L Q O O C0 UU0Q( Q L CX CUZX L U U1U Q ) 0 X(LXUUC QXC1 L
) (0 0 11OX) X0 C XXL U 1O00 Q( U O( )(Q?X0 1Z L X ZUQ 0 Z O UC CO/ C L0 UL 1 0 Q 0 ( O 1LLC ( X U(X O Q CL / L X 1 LQ1 0 ) 0 U L ( C ) C 0 U U C ) 0L C 1 L X1 ZQ X 0 O)ZZU? OQO )/L) ) O Q)C X U ?UXX( L0ZO Q( L(Z
U1C C ZXQ 0CO?C U? C / 10CCX0)Z X Q U Z X 01 1 L O ) L L O1CCL X 1XC X O U X1 ? L L0Z 0 1 1 1?O ) U 1 Z X / 1 L O ? Z X ( Z QO (X Z (0L 1 X// ? L UL1 Z1C X Z 1 O 1 U1( C CC0 )LC /QC) QOCO1 XQO0CC
QC1( QZU1)1 LO UUZ OU LU L U0 ?0 L C Z1 UC ( /U0U / 0 / /( Q L C O Z / / X Q) X 1 1 O0 1 ( X Z C ZZ L LC 1XLQ U ( CC0 ) QQ L 1C1/ Z ) Z Q1 ZQ 11 QU 1 ZX X L 1 Z/O0 ( ?U( CLX1 ZXQX O LZ U 0ZU)XXQ/) Q ? 1 QZU
LX0OL XCL1 0 O ZC1L 0QC/ Z Q ?C ( XCO QQ) 0LO Q )Z OZ OLO O 0 Q ( C0 X C 0U / 0L 1 1C Z ? L / Z 11 Z 0O Z L U (/ 0 / ZU QO Z O Q ? /X 1 ( 0 X Q CX 1 ? C X QL ( X ?Z C /Q O U?) LLQ10 ( 0L X X)LCX? ?C111U) QZZ O 00(00 O) U LCCZ
U 1UL L ?U /U/ Z L1 1QOO ULX0(XQ ) CZZ/ ( ) ?XQ0CX 1XL Z U 0 ( Q L ( ? OZ1 L 0O Q L0 U C U 1 O( XL0X 1 ?0 Q ) ZQ Z LU0 U O L Q X/Z C Z CO QCC U1L Q/ Q Z L C X L0Q Z 1 ? QUZX 1C /U Q U QZ C) XQOO/CU C X/OOUU UZ0O? C/CL1
) 0QC1U?UZZC L1 ?ULZ?XQLL L0 1) L X ZC /U 1Q0XL Z 1/ 1C U X 0 QU O0 ? C1Z/1 C L 0 O 0 X0 O 1 ? C X1 1 Q O Q( ) /0 U ZQU 1? ( ( 1 LO)1 C1 (X? ZU /U UU ?C Z0 CC CU O XQ0/0 ?(1 LZXU0 O U0?CCC(ZU 0Q X 0(UO1LL XQOC
0U X QULU ?Z1ZOO )0 Z0U?? LZ XUOXU0 LX)L ZL ) OO? O C 1 CQ? Q QZ Z X0LU UL LCZ 1 UQ Z0 OX X / O ( U C L C 1 1U OQ ) U ? X C OUZ C ZX X11 0 U 1UQ1U U 1Z ) ZL OX )L L Z U) C / U CO?1 /Q ( 0O ZZ 0 U LLZ?0X(1ZZ0ZX(
OL0UZLLQQ1O CU /01(CZ011L 1OQ 01 0? C 0 OU 0 /U Z1Z / Z O0 0 L )/ 0L C1C Z0ZLC/)CU01O O Q 0OZ X / OX X 1X LL / Q XZ1 / QUCQ1 OO ) 1 U? Z Z C U Z Z) L1 / ( L OCX/ ( CUQX ) C L ? 0 XZ CXU 0X X 1( CC 1/ Z? XU (QQ(0 CX0 U0(Q)0? CLL/UU1 O
CLCU CCUL X (0?CXOCCQZ11 10OQ/0O0Z11 C LXL ?CQXUO 0U1/U U LL /U? 1(U/U0 /ZC? / OL Z O1Z/0 ?OL 1 C ?10 Z)/ QX ( QC U ? Q XO CX X UUO U UULQ 1 Q ?X( Z 0) C Z CX ?C1 Q Q1/ Z 1 0 / ( O XO U )UZ Z O XXQ LL ( / Q 1Q L U QL (0 Q Z X((U O CLOXUQ UUC0LUQ/1)?ZQL
C00LQ)QU U C /CXQOUQX0 ( C( LX1OQZC)ZZ1U Q/1 CO? C OO0 QZ / L ?1)U ? /? ( / C 0U L) 1 X) X0 1L OU O Q ) L0 OC/ Z ?U Z OU /L U0 X / ? Q1 U L L?X 001 C U0 0 Q 0 O0OO 1LQZ UU Z?0 ))U ZU/XO/( /C0 OXXCC10 1 ((? Q ?XUXO)1Q1UC00U/ZX QXC1/)0U 0?/X010OCC UC1Z 1?U /
0C0CQQZZ 0QX(LZ CO1ZUL O/UQOC 1O XC ( L1( OUL (Q (U CQ C 1OXU U 0U 1 U 1L ZQ00 UO) ) C ? L 0 U 0 ? 1( 1 X 0 U1 U Z )XO L U)/ O)QL XOC X1O C1( ? (0 C1 0 ) Q)X? ZXO OX / X X10 11 Z00 X1U Q /L (0001 1 0 1ZXOOLL )L0 0 ) Q ? U1Z) X (U 0(U?QC1LXO Z0OZQO
0OXLZZ/OX0/X 1U) 1UC1CL?U0? C C?) ?/ OX X CX11( OO/C1 0U0X Z 1L Z (CUCC Z U L CQ X 1 C O0 O L Q00 CC Q 0 ( 0/UCZ )Q ? 0CX CU 0 QC ZXX 1? O UCL L U U X?0) O (XQL / UCO Q0 ( ZL Z L XOL Z Q L 0O (XCO/1LO 1QX QX ZX OOX1Z 1OCO 1Z X XLZ( C 0LQUCLXOZL (UZ1/Z XXL)(UOZZ1?)
X(/00O1O1)/(UX?01OQQQ(OL)1COL ZLZ OQZXX L ) QO Z Z O C CXX OXC ZC / 1 0 X/) Z0 QZ/C X0 0C10 CL Q C1Q11 ZZC/O /C QZ(O 0 Z 1Z ) 0CL/ C0 /0 )L O Z ?O OX1 ? OZ QC/ZCL 1L 1 O QX10 1 1 X ( XXXL (1Z 1 Z0 Z Z1 XU L ? 00QQQZ? L 0 U)CCL LQ00?1L1QO O Q/Q L OLZ Z) XOUUU(LOQ 1C LXOQQL)0CX0)
OL?C)CZQCLZXCU L? QQX (X CQ1//UQ(U0 L 1UC) 0 0O CCOUCC CU 1O? 0(O L L L ( C X U CQXOUQ ZLQL (ZX U( O C?CX UQ/1 0Z 0Z 0 C 10 ?0U XZ Q Q11 OZ) 0C CU C0 O ZU001L /)U Q X X ? QCQZX ZQC 1LO1O OU (0)CU U /C XLQOL1?UCOO QO0X(L/X 0L)1)C LLLZOCUO OX 01U QQZX ZL0O 0 ZZ)COOO0
UZUQ0QC)C? XQX ZO1LQZCCC1 UUU 0QC) LQ(C1/QQC O O O1 X LXL00?/ 0? QZ X0 Z U CO0 O) QL1 U1QX Z 0(C /UL?01 ZL0/ 0 ? 0 O U/UQ Z C Z OUO X 1OOO OL QZX )U 0 )L( U / U(ZO QZOL 00X1Z? X 0 CXC Q00 O L1Z U? LX ?O 0 1/ Q1 OX O C/ X U/0X (Q1 C QZ QZU QU 0? L 00XQUL0(X)U / LC1 1OXXOQ0C 0X
QCL0QZ))1QL1ZOCO /LUQC 1XC)X( (OX O OLXCQ? LUOZ 1 0C Q O/ OXZ1QOCLQZ /UZ LZ OQ Q LQ C U XLL C 0U UO X 11ULO00? QU/ 0U O ZZ 1 ) X ) QZ / Z O1Z/Z OX/ ? ? 1 ( )(O 1 X L LO ?0 X1L OXXQZ Q 0 10 1 )?1L 0 1UOL1 L OULX OX 0 0X 1L 1 LQL1 0X/XXC110L?0 ? )?0XLQ0 (C11 00COUC)Z)Z/Q0QCX?X0LZ
1OUC0QQXQ 1ZZ(Q1LL L1Z(?C?ULQOX UZCZZOZZQQ1O QU0 C0/ XQL0 CU X? ( /OX 0Z X)C Q ZZ Q ZX QX?O 0Z XU ZQ0 U0Q10 UUL L 1( L1 L/ ) XC U ? XL ) 0 Z )) X 1XQX ?U 1 ) (O OQ) X O LZZL Z? ZO Z Z?U1 1O L(X QUQ X L((LLUX 1 OQZ C (CQ Q LU L/ZC 0 QZ OC/L LX LOLQQZZLCZ) CC)UOLCZ0/Q1 XQ U0 U 1OUCX1XC0OXCQ
0CO(1UXC1U1)LL(001X?U (0UOL0CZU0UXO11) Z0U( OO1L CX1 C 1 1 (U U OU ? 1Z ZZ1X ZZX C L C OX1 XZXQ ?LC1 U) ) CU (?ZZ ? Q U 0C)U0Q/O Q )X0QU/L XZ OQ0CL 0Q O UU 0 U X ?C OC Z U?Q Q (UO0 ) (ZL) L OQ1QQ ( CL X X) C XQ XZOOO XC1X0X1O L Q) C C 1LX?QLLU1UC1QU1X0ZXUQZUZLQX X(1?U OO?O?XXQC)CLL(XO1/X/Q1QULC
ZQ1LZXZCCZZUZQ(XZ10X/L0/Q(1LX?X0Z X0C/ X)XC Z U/OQZ//?/QC1COX1ZL O L Q1O/OLU0UZ Z/)1 0XXC O/ CZZO (1 OL CU O 0 UOZUZ QOX L 0(Q U0O QOO /O10UXO0LOOO Q Q CU CQL L0OQ Q( ? X O?Q Q U O0Q ?LX Q C Q U?O)Z0ZO?U( 0 (ZQ / O1XZQX )/?0O Z/ ( )0 X X1?X ? Q X1Z1UUZ Q/ /X01((X1))XC0O10C0ZULU(C1CCLO0?
0X/)OCLOOQULCLXQ(OZOU(X00?X0C0? X)Q UL XOXZU(/CLO0OZ1/ XQ1OLCL0UQ C O0(1 XCOLQ (CQLQQ C01 / Z L/) 0 QL X Z 1OC1 Z1/UU L Q U)X1Q Q C XC/ Q/Z )(LU?U1L1ZX) ) X Q O1(10 U X O LU UC UU0 U /? Z LC ?)OLU1 Q L1 Z QOX 0L1QO0 00Z0?LC )OXL ) ZCXU Z(Q 0 ) Q(?O ZLL 11/X L1U??UL)X(U )1/O0/QXZ?CLC Z?0ZCUU/U(()Z
L1L11Q/C)XLC/CLZLL0QXZ(QXCZ?U0)OXCO0( OO Z 10 CU/QX(0 O ?C1(/10)X0O UQZUU ZCL QOOQ0X/ / 0Q1 (QCXXL L ZQZ OC?OCZ C0( U ( C10 L Q 1() XO /QL 11 L(CO C 0 OQX X CZO1 )C 101 (0/ ) ZU C0 UL11L)0QQ1Z0O XUC/ ZXXLZ 0QXQOO X 0CO L1 1QUQ1Q 1 Q(X?1 LO/Z ZZXCQCZ(O0XOL0QC QCUUXXUL0LZCUCLCZ0OLU0Q?0O)UOO
)?C11CZOOQC)/QQLO 11CXL0OCOQXXZX1LZC/QU1CO1)C///10X LQQQOL? C 1X QQC1CQ)(ZU((CQU 1Z0 ZXZ0 UC UQO C L0U)/Q) (X Q ) LCXC Q0 (1 U( CLQ? Z ZU Z0 Z (Q 1Q/) ?ZOO /Z0 Q1) L ZLLQ 0U UU C Z/L 1 LC 1O L0C 0LZ OLULL 0U 1 C (QQX0QQ CCO0?CXZ)0CLQ) U?OQU1C LLU?C1?OZ?QCCOZ0L10X0QLZXLLLQZQO(Z0(/)1XUZXZQQCLUQ X1LC
1O/L(1/QLL0LZULQXZZUQ0Q1CLU1?QL C1 0ZXLZL1/Q/L??QOLOQ?OULX1)L0QOUC0LL ? X1?O CXLX/ QC 0 L 0L QULX1U C X X(X 1OXO / L1Q LXZ0 L1 X O ) 0ZQX? (ZCQ?UO ZLO XXQX Z ZXQ (1O Q 1 O ) CU1Q X 0QC)X/0 XX CCX) Z OQCU)UQ /Q XX/(Q Q U0C LXQC OLU ZZC ( LUCZ (LQOC0 CU(ZO 1XXZC/XL1/UZ01U?C 0ULQQXUQZZLUUXZ?0O XCULQZL?OU0
U11OU)01(LUZO)L01Z1/Q1Z?XC(CU0(/)CL?1C0C LO)(CLL/LU( C0ZX0ZZZLZU?O1 LZU01X QO0 1U/L 0/ ?Z 1 U1U Z L LCUX01QX10ZQX Z1 ?CQLQ OCC1 XZCZZLZ(X 0 1X1 0ZO OXZ1 1CL Z10)LQL XCU? O1 LUZO L Q ZLOQ 10UOQC U C L L QQQ? XQ/0X L)(C O)OZO O(/?Q1Q? Z XLUC QQ U1 ZL1C1ZXUXC1Z(LZU)ZZUUX/L11?L QO?/ZQLCCUOC0CUC0O(1LC)??0L U
L CXQ/Q U0?(ZCLC ()Z(X/1ZZOLZ1Z(UQZC0/0QO(L U/) (O/ Z (?ZZX)( X ZXZ ZLOX/Z? XO1 LCX1U OZ 1L(/Q/ 1Q OO1 Z XO( U 0XL) XL 1? O/(QX LUOL Z L 10U) O L1 Z? LL/ ?L C? /0 LZLOL C 1Z) L /0 Q UU Q Q/ X XU(/OC( ))ZUZ U )0C0 ? LC(U QC Z(QC0 0X0Z 1X0 ZL/X1ZU1)0C?O1U0LU1X?LL0U0QZ/1QCL0UU O Z 0CXOUL?XZ0UCXQ0)Z0
LZZ1LLL?L00OZ QL1/QUCZUO?1QX01U )Z?0Q/XZXOZ1U00C?U?XLQ QQLZ1LLZCZO X(0OZ1ZZ(X)L1Z 0O/ X0 )?U(O0? ? )1QU10O1CQC1 UL(1 ? X)X X1 C )LQ)? Z/ 0C/(QUQO Q U C ZCQ( L0X0 U0LL ?/ UX/OLUCU /LZ O)1 LUL 1 X L L1C0)U ? CC 1LL XXC)QQ//C000UC CLUX1(1 QOZX( 10C(11CQQQUQO()XC(0UCUXCOOXC? CO1ZXCXL/ZO00CZUZL? O11QU/ZOZ)0Z
1Z?LLOZOL?))C)Z0XX Q LL?LQQQXL())ULO1LCCO1XX0UCUC(1/ZOXO10(C/ZL QZ?OXU1(LZQX QUXCQOLZ/LC/CX))Q0OL?/Q ?OO 1U XC10 0 1L( 0Q 1/Q LLU)(Q 10 /1L0 0)L/1CQ(XQOQLX)L CU) C1Q0LCX L)))Q)LOC 1X1QL Q?Q XLOC OQ0X LZQUXO CCLQ ?0 )Z/O(ZLUXZZ OO10L U1OO)O(10CQ0 /QXQX0XL/QCXUUCXQU0LL0 )ZUCO01OU?OO1QOQ)OCX)(CQUOCL(/LZC U?1U
UULZZ?CZZ?ZZOCLZC)001 / U1L1X?111)OQULZ XUZ?X?/0( Z X1 C(XU)ZU0 ()CUL1LZ0)L? 0Z /UZZ?X1XQO0LZZ1UX LC1ZXZUOXOO C ?0U UXL?ZQ )00OLQ1CUU?1Z?ZC?(Q)QL ?? UX/QO/O(Q010Z /LQ1ZZXCX C0L UXUC C/C Q UQUCCLQ? U1Z))/QQZXQOQLZCXLU CQU0OU? XZ/1 ZQOOUZOQOQQQL (OQ / X( 010Q101(Q?/LZOX1(1Q/CZQ CZQCL1/QO ZUU 0UUC)U ?CXCC00OU Q)UCCC
(LO?/O??XXXZ/110U0 /0QQ0U1CQZ LL1? (X0XCZL1U?)10U/)0( )0UQ LQUUOX)/X0U/1 X/)0ZLXC XL1/?ZQULZX0?LCZUO0ZLO?L?U)QU1(XOCX CZZ0 L XUX L / )Q(X 0 0O)1QL11 / 0 ?XXOZ)/Z CZU1QU X// L 1QL(1O O Q 10)C1U1CO? ZXZC/Z1O1C C0ZQ(UXCZO0CO1XQ//)QOOX(1/XQ/LZQC))1LZO0UQUX?LC0LXZ(ZZL10ZLO0LQ L1)C1ZZ1ZQ ZZOL/L/100 ZXL OQZZOZU OLOUQ)(
)OUXQ1Q/(OXXOOQCLOC 1CC1)OOUO/LZ11UCUUL 11XXQ)Z0/CZUUUO(/C0?00Q?QC)OXXZL ?OZZ//1 CXUCLZX0Q0)1 U L ) 0U11( OZX 1UQ UOOZX?Q1C)O ()ZLZQ1C0 OZ 0/0ZZQ Q ( ?ZZL1L? 0CX CZ)L 1O 1(LZX ZLZC0Z Q) UL L)OX X U Z OU0 U LZX1/QOUL ZOOQ1QX)/Z?X(ZUOLUU(O0(0?Q1LZ1/O0)CZQC/1Q(CLOXOQCUCZQ1U OC?U0 OC(U1O 00OXOL1C0XO) Z1XZ1XOUL
LUL(X1 C? QX?0?Z)1OOXZ) U0 X(Q0QUU1Q() OZCXC)0O1ZX/U11LLQ00/?QQ1C?C0)0ZLXLLX1OLO(1LU)LOZQL0QZOU Z/CXQ)UUXZ0XQ CZZU(OQO UQ0U0LOC0 ))? 0 (0 CO0XL0C0L11 1)X)1QUQ1 1X0 L 1OX11ZL/UZULX/0O0QZ0111? Q/Z)/UQ )ZU11 QQ /Z0XZO1O0QZOZC/0U0 C( XX1XQ?CO 1CQ1ZC)0X(ZU)/1QL/0UOL?(/QC1(QZ(XLZQXC(U1?QCU1L0Q0COXLC0U(XX1)ULZ?ZQ0XCUXZ?Z(
X/010XUQO1)OL01(OXOZ(CX0L0 ?0 /?QCL(LX1U(/ Z1XZZ()1UOU)XO1QLO1Q)QZC110?1XZQ1C?LX1O 1?QZL )O1UL0/1 CZU)/)QZXUX)UOCZ(XOLQ O LL/01QXOCO X11OXZ 0C0U 11/LOCQQ X0XO Z01UX Z110C?CU1?CXLX/Z1ZU11U OZL0O)OZLZUC0 100U0X1/O0LL/? 0C(QLLOUX/?X01C?LCC0 UQC1OLL(QUUXXOZ?/0ZXQX1 /LLQZ11XX1C( OZ OZULU?ZZ?OC)ZX)XLZQXX1CO00X0)01XLQCXL QZ
BENCHMARKSWe test frontier models against human-verified data drawn from real conversations and real work — the conditions models actually meet in production, not curated test sets.
Fifteen frontier transcription models against real multilingual conversation. Best-in-class still misses every second word of dialectal Arabic.
In progress.