🟢Subheading Benchmarks
Dataset name: Subheadings_OneAI_v1.3
Dataset
Name: Subheadings_OneAI_v1.3
Datapoints: 32,561 (tested & reported), 10,000 (published)
Sources: Various sources for news & blogs from various domains (news, politics, tech, finance, etc.)
Results Overview
| Grade | model | Rouge1 | RougeL | RougeLsum |
|---|---|---|---|---|
| 🟢A | Ours (One-AI-subheading-v1.7) | 0.469 | 0.464 | 0.464 |
| 🟠F | SOTA open-source model 1 | 0.186 | 0.178 | 0.178 |
| 🟠F | SOTA open-source model 2 | 0.133 | 0.044 | 0.127 |
| Did we miss a model? suggest one and we'll add it |
Note: see our overview on the Rouge metrics
Detailed results
Ours (One-AI-subheadling-v1.7)
SOTA open-source model 1
SOTA open-source model 2
Updated 9 months ago