T10 Follow-up in an old thread
Category: Large. Pass at 2/2 per thread.
Protocol
Day 0: run T06 (CSV) in thread A. Keep the thread. Day 3: send in thread A: "Add a 7th category 'Other' for rows whose note is 'reimbursed', recompute the totals." (= follow-up in the old thread) Day 3, in another measurement slot: new thread B, upload the same transactions.csv and send the T06 prompt plus the follow-up sentence above in one message (= reference in a new thread). Measure: tokens(follow-up A) vs tokens(B). Report the ratio.
Inputs
Uses the inputs of T06.
Checklist
Pass at 2/2 per thread.
- The new totals are right (answer key, made by the same script)
- Category 'Other' holds exactly the rows with note 'reimbursed'
Main measure: tokens of the follow-up in the old thread / tokens in the new thread.
The answer key is not published while the task set is in use.