limited topic trial  is on hold for most of 202122
The big picture question
Does teaching with timely practice embed significantly more learning than teaching following ourcurrent scheme of learning and program of study?
The limited topic trial will
 measure the learners learning gain from teaching
 using the school's current scheme of learning,
 using a bespoke timely practice scheme of learning, which replaces another part of the school's scheme of learning.
 teach the teacher how to use the timely practice app.
The limited topic trail will be free for:
 training,
 creation of scheme of learning,
 use of timely practice app,
 use of teaching resources;
but we may charge for
 travel, accommodation and sundry related expenses if we need to visit your school.
same teaching skills
The skills required in teaching a traditional maths lesson and a timely practice maths lesson are the same:
 the ability to help learners hang new learning on existing learning,
 clear explanations,
 knowing when to use closed and open questioning,
 give effective feedback and
 empathy.
The difference between teaching traditional maths lessons and timely practice maths lessons are in timing
different timings for teaching and learning
traditional maths teaching  timely practice maths teaching  

scheme of learning 


start of lesson 


teach new learning 


practise new learning 


practise prior learning 


feedback 


after feedback 


assessment after learning 


different timings for assessment
Depending on when we assess learners (especially low attaining learners) we will see a very different picture of how well teaching has become learning.
For low attaining learners (those not expected to gain a grade 4 or higher at GCSE)  
time elapsed after teaching  traditional maths teaching  timely practice maths teaching 

at the end of the lesson 


next maths lesson 


a month 


3 months 


to measure embedded learning we must delay summative assessment
Condition A: For an effective trial we must delay testing for at least 4 weeks, (ideally perhaps 7 weeks  a summer holiday's break) after teaching / any revision work/ any end of unit tests which the school normally uses.
pre assessment must be robust
From previous trials we know that one question asked and answered isn't sufficient to decide whether a learner already knows a skill or not. Only asking one question makes calculating learning gain open to teachers judgement and to luck (see the pre assess must be robust box at the bottom of the page).
Condition B: For a simple yet accurate pre assess use timely practice pre assess assignment and avoid false positives by asking a second question on every layer which appears to be known via a timely practice classwork assignment. In the trial the assessment choices for pre assessed layers (bites of learning) are perfect or not.
Condition C: We must keep the teaching and learning in the business as usual (BAU) and the timely practice (TP) lessons separate. Therefore each learner will have two timely practice profiles, one for all the topics which will be taught in the the BAU lessons and one for all the topics which will be taught in the TP lessons.
 The BAU profile will find and verify the layers (small bites of learning for a topic) which the learner "already knows" in the BAU units and will be used to create the "delayed summative assessment"  which will assess both what the learner already knew and the new learning taught in the BAU units.
 The TP profiles will find, verify and then more deeply embed into long term learning what the learners already knew in the TP topic themes, ensure that the new learning in the TP lessons is practised at intervals to ensure retention and to assess that learning in the "delayed summative assessment"
make the trial as easy as possible for teachers
 disrupt the teaching of maths as little as possible  so the teacher will teach the BAU units from the existing scheme of learning, programs of study and resources
ensure mastery learning is used for the timely practice lessons
 teach on firm learning foundations. i.e. the pre assess must be used to plan teaching
 the teacher must teach only one layer from each topic
ensure practice questions are timed to efficiently embed learning in the timely practice lessons
 the learner should do less practice questions on the learning of the lesson directly after teaching (this is fewer practice questions than the teacher would normally do directly after teaching),
 the learner must do retrieval practice questions at close to the optimum times calculated by the app (i.e. a timely practice assignment must be done every maths lesson for at least one month after new teaching and twice reducing to once a week within the second month after teaching),
Condition D: For the timely practice (TP) lessons the teacher should follow the scheme of learning which timely practice will create after the pre assess process. This scheme of learning will
 teach all the prioritised topics from the existing scheme of learning along with most of the other topics from the topic theme,
 ensure that the fundamentals of mastery learning and retrieval practice are applied.
Condition E: For the correct timing of the timely practice questions, the learners will need to do a short timely practice assignment for 5 to 10 minutes within the subsequent BAU units. The teacher will need to give feedback during these 5 to 10 minutes sometimes to some learners. The 5 to 10 minutes of timely practice and feedback can to some extent replace the warm up of the maths lesson. To compensate for the fact that timely practice (TP) business is taking up BAU lesson time, the duration allocated to teach the BAU units must be extended and this time must be taken from the time allocated to teach the TP topic themes, (TPTT).
mimic a tightly spiralled scheme of learning in the timely practice lessons
A 1 term trial means the teacher can only teach each topic once. The teacher may want to return to teach more on these topics after the trial is finished.
A 2 term trial means the teacher can return to teach more on each topic in the second term. This will give a clearer picture of how timely practice works.
Condition F: The school may choose a 1 term or a 2 term trial. The school may hedge their bets and decide whether to continue the 1 term trial into term term 2, after the first term. If this is considered a possibility, then the we must plan in advance which units will be taught in the business as usual lessons over the two terms, prior to starting the 1 term trial. See Condition C.
ensure that it is decided in advance which topics are BAU topics and which are TP topics
There are 5 topic themes: number, word and proportion, algebra, geometry and measure, and probability and statistics.
Condition G: The BAU units will come from 2 topic themes, the TP topics will come from the other 3 topic themes (at least 2 if not all 3 topic themes will be taught). There must be a strict separation of topics, to ensure that the learning gain the trial calculates can be attributed fairly between the BAU and TP lesson time.
Summary of conditions
Condition A: delay final testing for at least 4 weeks/ ideally 7 weeks after teaching / any revision work/ any end of unit tests which the school normally uses.
Condition B: in pre assess each layer must be answered twice correctly, to count as "already knows"
Condition C: we must ensure that the BAU and the TP lessons do not overlap teaching content, so each learner will have a BAU profile and a TP profile for assessment purposes.
Condition D: BAU lessons are taught as normal except for condition E, TP lessons are taught following the TP scheme of learning  which will be created after the pre assess process
Condition E: 5 to 10 minutes of most BAU lessons will be used to review TP skills, so BAU units will have an extra lesson (or two) per unit and TP will have one (or two) fewer lessons per unit
Condition F: The school may choose a 1 term or a 2 term trial.
Condition G: The BAU units will come from 2 topic themes, the TP topics will come from 2 or 3 topic themes  there must be a strict separation of topics.
Decisions for this trial
For conditions C and G we must split up the maths content (topics) to be taught into BAU and TP.
The BAU units must be selected from only 2 topic themes.
The TP units must be allocated the the other 3 topic themes. timely practice is hungry for topics to teach, because we use a breadth first SOL how this impacts on the trial is explained in more detail below the table of topics found in each topic theme.
The school can choose which of the units in the period of the trial will be BAU will be taught in term 1 (and which might be taught in term 2)  these must be drawn from only 2 topic themes. An example follows on how this might be done, but first we'll look at
Which topics are in each of the topic themes
Which topics the learners might be ready to learn
Number  FDP  Algebra  Geometry and Measure  Probability and Statistics 



 
Number  Word and Proportion  Algebra  Geometry and Measure  Probability and Statistics 
Very low attaining learners are likely to be ready to learn just a few topics from each topic theme ...  
e.g. only ready to learn 6 topics  e.g. ready to learn 8 topics  e.g. ready to learn 5 topics  e.g. ready to learn 6 topics  e.g. ready to learn 3 topics 
... whereas some older and more highly attaining learners may be ready to learn considerably more topics from each topic theme,these topics are prefixed with the word "perhaps" e.g. "perhaps BiDMAS" means that it is likely that a number of low attaining learners will not be ready to learn any of the topic "value of: index" in the trial. There may also be some topics which learners no longer need to learn e.g. "solving ready or perhaps solve" means the learner will no longer need to learn solving ready once they are ready to learn solve  
perhaps up to 13 topic  perhaps up to 15 topics  perhaps up to 12 topics  perhaps up to 13 topics  perhaps up to 9 topics 
Why does TP need 3 topic themes?
As already mentioned
 the BAU units must be selected from only 2 topic themes
 timely practice is hungry for topics to teach, because we use a breadth first SOL
For a very low attaining learner  if only two topic themes are chosen
 e.g. Algebra and Probability and Statistics the learner may only be ready to learn 5 topics from Algebra and 3 from Probability and Statistics,
 e.g. Word and Proportion and Geometry and Measure the learner may only be ready to learn 8 topics from Word and Proportion and 6 topics Geometry and Measure,
in both cases a further topic theme will be required to give the teacher enough to teach.
For learners at the lower quartile in KS4  two topic themes may well give the teacher enough to teach
 e.g. if Algebra and Probability and Statistics are chosen, the learners may well be ready to learn 12 topics from algebra and 8 or 9 from Probability and Statistics,
 e.g. if Word and Proportion and Geometry and Measure are chosen, the learners may well be ready to learn 12 to 15 topics from Word and Proportion and 11 to 13 topics Geometry and Measure,
then perhaps a further topic theme won't be required. However if there is even one learner who needs to be taught more topics (i.e. from a third topic theme)  we want to ensure that the trial doesn't hold that learner back. Hence we want timely practice to have 3 topic themes to choose from, meaning BAU must be restricted to 2 topic themes. Which is why
 the BAU units must be selected from only 2 topic themes
 and the TP units must be allocated the the other 3 topic themes.
This sample SOL is from KS4, see below, where the first 2 units will have been taught before the trial starts.
 2 of the units from units 3 to 12 must be allocated to BAU for the 1 term trial,
 4 of the units from units 3 to 12 must be allocated to BAU for the 2 term trial
but these must come from only 2 topic themes.
A colour coded Key is used Number, Word and Proportion, Algebra, Geometry and Measure, Probability & Statistics
One option would be  
BAU unit  TP topic theme  
Unit 3: simplify and substitute  1st term of trial  this can't be taught 
Unit 4: angle rules  1st term of trial  this can't be taught 
Unit 5: averages and graphs  this can't be taught  Probability & Statistics 
Unit 6: FDP and ratio  this can't be taught  Word and Proportion 
Unit 7: sequences  2nd term of trial  this can't be taught 
Unit 8: data graphs  this can't be taught  this is already chosen 
Unit 9: shape properties and calculating space  2nd term of trial  this can't be taught 
Unit 10: equations  AFTER TRIAL  this can't be taught 
Unit 11: transformations  AFTER TRIAL  AFTER TRIAL 
Unit 12: probability  after the trial it may not be necessary to teach this  
return to Unit 1: Calculating  Number 
The TP topic themes are Probability & Statistics and Word and Proportion and Number
Another option would be  
BAU unit  TP topic theme  
Unit 3: simplify and substitute  this can't be taught  Algebra 
Unit 4: angle rules  1st term of trial  this can't be taught 
Unit 5: averages and graphs  this can't be taught  Probability & Statistics 
Unit 6: FDP and ratio  1st term of trial  this can't be taught 
Unit 7: sequences  this can't be taught  this is already chosen 
Unit 8: data graphs  this can't be taught  this is already chosen 
Unit 9: shape properties and calculating space  2nd term of trial  this can't be taught 
Unit 10: equations  this can't be taught  this is already chosen 
Unit 11: transformations  2nd term of trial  this can't be taught 
Unit 12: probability  after the trial it may not be necessary to teach this  
return to Unit 1: Calculating  Number 
The TP topic themes are Probability & Statistics and Algebra and Number
This sample SOL is from KS4  it is heavy on algebra, but light on number, so TP may need to teach a third Number topic theme to the lowest attaining learners. This is likely to be a review of the first unit of the year, calculating. However if all the class are ready to learn most topics in the two chosen topic themes, then only two topic themes will be allocated to TP.
Assumptions for this example trial
FYI we can devise a bespoke trial to suit a schools unit length, lesson length and term length.
Teaching
Assuming in the 1 term trial:
 4 units are taught in a term
 each unit will be taught for 3 weeks
 each week has 4 maths lessons and each maths lesson is 1 hour.
i.e. 12 week term with 48 maths lessons.
For a fair comparison, TP will need to use 5 to 10 minutes of each BAU maths lesson (perhaps replacing the warm up), see Condition E, i.e. 60 to 120 minutes, i.e. 1 to 2 extra lessons.
So each BAU unit will have 12 + 2 lessons and each TP topic theme will have 12  2 lessons
BAU will have 14 lessons to teach each unit
TP will have a total of 20 lessons  this may be split into 2 units or 3 units (depending whether the teacher will teach 2 topic themes or 3 topic themes).
Assessment
Accurate assessment requires verification with timely practice
We need to accurately measure longer term learning gain. This means we need
 an accurate measure of what each learners "already knows" and
 to wait at least a month (but ideally half a term / the duration of the summer holiday) before we do the "delayed summative assessment"
to be sure our calculation of learning gain is accurate as possible.
timely practice makes this easier than it could be, however the design philosophy of timely practice is slightly different  timely practice uses data to ensure that the learner embeds the learning for the long term, so timely practice does assessment for learning  whereas for the trial we want to use timely practice to do "delayed summative assessment".
The main thing we need to do is ensure that the pre assess data is accurate  meaning each learner will do a second verification question on any layers, the assessment of the pre assess assignment showed they "already know".
pre assess must be robust
Many low attaining learners have non predictable learning gaps  so what to a maths teacher looks like two similar questions (a) and (b)
 one learner may find (a) easy for them but (b) too hard for them and
 another leaner may find the opposite (a) too hard, but (b) easy for them
Learning from the past
When we have measured learning gain in the past, we find that after asking only one question on a topic we may think the learner already knows a layer  but by asking two more questions on the topic  we find that they do not. We refer to this as a false positive. In our intervention trial we used a points system, one point for partially knows, two points for fully knows  but this was arguably open to interpretation and made measuring learning gain complex. For some learners up to 50% of the layers we tested gave false positives, for other learners the proportion was negligible.
Applying to the future
For the limited topic trial, we will require every layer to be asked and answered independently and accurately twice before we judge the layer as "already knows". This will make it easy for
 the teacher to assess each question (perfect or not),
 the results of the trial to be analysed (one point per layer  no drilling down to find the learner's past performance on answering each layer),
 the teacher to plan teaching during the timely practice parts of the trial (teacher sees only what the learners already know well),
 the teacher to give feedback during the timely practice parts of the trial (not wasting time giving feedback on what it turns out the learner doesn't know).
It will also make
 it fair when comparing the learning gain with the BAU taught topics and the TP taught topics,
 easy for schools to use the assessment for learning data from the BAU taught topics should they decide to continue to use teach timely practice to teach.
Advantages of using timely practice for assessment
 Assessment is quick, as the app presents the answers to the teacher, together with the assessment code options for pre assess the teacher has the choice of or the app then collates all the data  and can easily create a second assignment including only questions the learner seemed to already know, the verification assignment.
 The topics are split into small bite size pieces of learning  which is ideal for low attaining learners. Partial learning can be measured. For example we can see which learners can write down a coordinate in the first quadrant or the first, second and fourth quadrant or all 4 quadrants. As teachers, we already know that to take a low attaining learner, from not being able to write any coordinate accurately to being able to write any coordinate in any quadrant accurately is "a very big ask". However finding where each learner is on the "writing a coordinate accurately ladder" and moving them one rung up the ladder  and ensuring they don't slip back when we aren't watching  means that we can return at a later time to teach a little more. This also means small learning gains can be measured.
 The teacher doesn't need to collate the data  calming though filling in a ROG spreadsheet can be, it probably isn't the best use of a teacher's time.
 Learners can very rarely copy, as they very rarely get the same questions  therefore assessment doesn't mean moving all the chairs and tables!
Disadvantages of using timely practice
 Teachers can't use the "mark a page at a time" method for assessing tests  as each learners assessment assignment will be different.
Finding out in fine detail what each learner already knows, and what they have learned is time consuming
If we are to do a trail and get accurate data then there is no way around this. In the trial we need to find all the learning which is fuzzy or partial (and with low attaining learners this can be a lot of their learning), hence the need for verification, which increases the time that assessment takes.
On the plus side, if the school goes on to use timely practice  all the data collected can be used  and the classes in the trial can skip the Finding and firming learning foundations phase and go straight to the Teaching on firm learning foundations phase.
FYI timely practice is not normally as time consuming when collecting assessment for learning data, as it will be for this trial; because timely practice is in the business of embedding learning rather than measuring how much learning it can take the credit for and how much learning prior teaching can take the credit for.
What's a snapshot
After the verification question has been assessed, we can take a "snapshot" of each learner's learning profile (print out or copy and paste on to a spreadsheet) to accurately record the assessment and then merely count the "Number of layers correct" for each unit or topic theme.
What time will be required to do the assessment?
BAU units (business as usual)  TPTT (timely practice topic themes)  
Pre assess  up to 1 lesson per unit  up to 2 lessons per topic theme 
Verify pre assess  up to 1 lesson per unit  no extra lesson time  all time will come from timely practice allocated lesson time 
Post assess  as school's existing SOL  none 
"delayed summative assessment"  up to 1 lesson per unit  up to 2 lessons per topic theme 
Why does timely practice need more lesson time to do pre assess?
If every learner is given a test on every layer (bite of learning) which any learner may know from every topic in the BAU unit this could be
 4 questions from each of 5 topics i.e. a maximum of 20 questions in the BAU units
If every learner is given a test on every layer (bite of learning) which any learner may know from every topic in the TP topic theme this could be
 4 questions from each of 12 topics i.e. a maximum of 48 questions in the TP topic theme
How much time is needed for assessment for the one term trial?
 the extra time to assess the BAU units will be up to 6 hour lessons
 the extra time to assess the TP topic themes will be up to 6 hour lessons
Although very low attaining learners will require much less, probably a total of 3 hours.
How much time is needed for assessment for the two term trial?
 the extra time to assess the BAU units will be up to 8 hour lessons
 the extra time to assess the TP topic themes will be up to 6 hour lessons
Although very low attaining learners will require much less, probably a total of 4 hours.
Calculation of learning gain
Heres how we propose to calculate learning gain.
Calculation of BAU learning gain
GAIN = Number of layers correct in formal assessment of BAU1
PLUS Number of layers correct in formal assessment of BAU2
MINUS Number of layers correct in BAU1 pre assess
MINUS Number of layers correct in BAU2 pre assess
Calculation of TP learning gain
GAIN = Number of layers correct in formal assessment of TPTT1 (timely practice topic theme)
PLUS Number of layers correct in formal assessment of TPTT2
PLUS Number of layers correct in formal assessment of TPTT3 (if appropriate)
MINUS Number of layers correct in TPTT1 pre assess
MINUS Number of layers correct in TPTT2 pre assess
MINUS Number of layers correct in TPTT3 pre assess (if appropriate)
The trial itself
Please refer to the one or two term limited topic trial scheme of learning page
FYI
pre assess must be robust
Many low attaining learners have non predictable learning gaps  so what to a maths teacher looks like two similar questions (a) and (b)
 one learner may find (a) easy for them but (b) too hard for them and
 another leaner may find the opposite (a) too hard, but (b) easy for them
Learning from the past
When we have measured learning gain in the past, we find that after asking only one question on a topic we may think the learner already knows a layer  but by asking two more questions on the topic  we find that they do not. We refer to this as a false positive. In our intervention trial we used a points system, one point for partially knows, two points for fully knows  but this was arguably open to interpretation and made measuring learning gain complex. For some learners up to 50% of the layers we tested gave false positives, for other learners the proportion was negligible.
Applying to the future
For the limited topic trial, we will require every layer to be asked and answered independently and accurately twice before we judge the layer as "already knows". This will make it easy for
 the teacher to assess each question (perfect or not),
 the results of the trial to be analysed (one point per layer  no drilling down to find the learner's past performance on answering each layer),
 the teacher to plan teaching during the timely practice parts of the trial (teacher sees only what the learners already know well),
 the teacher to give feedback during the timely practice parts of the trial (not wasting time giving feedback on what it turns out the learner doesn't know).
It will also make
 it fair when comparing the learning gain with the BAU taught topics and the TP taught topics,
 easy for schools to use the assessment for learning data from the BAU taught topics should they decide to continue to use teach timely practice to teach.
About prioritising topics
We recommend schools follow their existing scheme of learning, so these are only sample units.
In most schools a topic theme will be taught in 2 to 3 units during the year.
Here are some examples of what topics may be taught in some a few units.
Number  Word and Proportion  Algebra  Geometry and Measure  Probability and Statistics 
BAU unit e.g.  BAU unit e.g.  BAU unit e.g.  BAU unit e.g.  BAU unit e.g. 
calculating  equivalence FDPR  simplify and substitute  shape properties and angle rules  data graphs 
BAU unit e.g.  BAU unit e.g.  BAU unit e.g.  BAU unit e.g.  BAU unit e.g. 
types of numbers  calculating with FDPR  sequences and graphs  transformations and calculating space  averages and probability

BAU unit e.g.  BAU unit e.g.  
word problems not suitable  pre assess takes too long  equations 
Once a topic theme is allocated to TP then the topics in the unit it replaces the topics from the unit may be prioritised, if there might not be time to teach all the topics from the topic theme  with the proviso, that topics will be only taught if the pre assess process says the learners have sufficient pre requisite skills that we can be (relatively sure) that the learners will retain the learning.