You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Brian Hauer edited this page Nov 19, 2019
·
2 revisions
The following is a work-in-progress feature for the TFB project. This page will be moved within this Wiki if and when the feature described here is completed.
A single composite score for frameworks
Coincident with efforts to measure and provide a score for hardware performance, we will be providing a new composite score for frameworks. The Wiki entry for TPR contains several details about how scoring works, so it is recommended that you read that first.
Below is a work-in-progress rendering of composite scores from Round 18 on Citrine.
Objective
The objective of a composite score is to give a single, unified view of the relative performance of frameworks across all of our test types. While the value of this score remains to be determined, we feel it is relatively low effort to provide. We will see how the community consumes this and adjust as needed.
Limitations
Only frameworks that implement all of our test types will be included in the composite scoring chart.
Scoring
See the section on scoring within the description of TPR for details on scoring. In brief, each Round will compute new weights for each test type. These weights will be applied to each frameworks' results per test type and then summed. The resulting sum is then divided by 1,000 to yield a shorter weighted score.