Tweet
Hi all,
When dealing with witness statements we're often confronted with information of an unknown reliability. Given my interest in trying to re-create simulations of the events surrounding the different crimes, one of pieces of information I have to use are statements a witness makes about some duration. Things like when Levey and Lowende each state how long they waited after leaving the club at 1:30 due to the rain. Levey estimated 3 or 4 minutes, Lawende estimated 5. Or, when Brown estimates how long after he got home with his supper did he hear the noise in the street of the men running along Fairclough (he estimates 15 min). These intervals can be used to try and string events together.
How good that stringing together is depends upon how reliable estimates of temporal durations are. When working on the simulations, I came across a research paper where they had people estimate the duration of various events that in actuality could span very little time (under a minute) out to things that last over an hour. What they found was a tendency for people to overestimate durations under an hour and after that to tend to underestimate the duration.
What I needed, though, wasn't something that allowed me to predict how much someone would overestimate a known duration, but rather, to take an estimated duration and convert it back to a more reliable actual duration. Fortunately, the article included enough information that I was able to use their data to come up with the sort of conversion table I needed (below; basically, look up the duration the witness states in the left most column, and the best guess actual duration is the average time, although due to how rubbish people are at this, you could expect the real duration to be between the min and max durations).
While I was reasonably happy with this, I am always a bit sceptical about conversions like this until they are verified. One of my research students is looking at incidental memory formations. Basically, people are shown 120 words and they have to decide if the word is of a living or non-living thing (some words they see once, some twice, and some three times - with a minimum of 10 words in between repeitions), then they do a really boring task for 5 minutes (they watch a clock hand tick around a clock and over the course of 5 minutes, 10 times it jumps 2 numbers, like from 1 to 3 rather than from 1 to 2, and they tap a key to indicate they noticed it. Honest, it's like watching paint dry. After they do that, we show them 240 words, 1/2 of which are the words they saw initially, and they rate how confident they are they saw that particular word. From this we can get a measure of how strong their memory is, and unsurprisingly, the more often a word was repeated the stronger their memory for it was. What was suprising is the fact that people are actually pretty good at it given they had no idea they were going to be asked to recognize the words from the living/non-living task (they get over 70% correct for words they were only shown once, and it just gets better from there).
Anyway, after they do that recognition memory task, they are asked to estimate, to the nearest minute, how long the study took them. They were also asked to estimate what time it was to the nearest minute. Again, they were not aware they would be asked about these things. She's collected data from 65 people (well, 67, but one joker estimated it took 111111 minutes, and another misunderstood the question, so I've removed those two). The average estimated duration people gave was 28m 30 seconds. If I look up 28 minutes and 29 minutes as estimates in the table, the two averages are 25m 28s and 26m 30s, so one would convert an estimate of 28m 30s to 26 minutes. And the average actual duration was 26m 30s! That is pretty good in my books!
The shortest estimate given was 9 minutes (which would suggest a maximum of 24m 43s, their actual time was 26m 42s) and the longest estimate was 47 min (which would suggest a minimum of 24m 58s, and their actual duration was 26m 42s). So while our shortest estimate did fall a bit outside the 95% confidence interval, we do expect that to happen 5% of time, so that's pretty good too.
Obviously, there is no way to take a given statement and recover the exact true real duration, all one can do is go with the average and consider the range. However, as you work with more and more statements, some times the average will be too long, sometimes too short, etc, and the individual errors will tend to cancel each other out. If you have multiple people estimating the duration of the same event, you can also use probability distributions, overlap them from different estimates, and from that try and get a more reliable estimate than you can from a single person.
Given people had to book a time to do the experiment, their estimates of the actual time of day were pretty good. However, it was interesting that out of the 65 people, 3 of them thought it was an hour earlier than it was, and one thought it was an hour later! So just over 6% were still massively wrong in the time despite having come to an appointment they booked the time for!. I rather suspect, though, that under normal circumstances if we were to just randomly ask people what time they thought it was, we would see a lot more similar errors (basically, people's error for the Time of Day was related to how far off their estimate of the experiment's duration was, which is probably not too surprising).
Anyway, while it's not intended to suggest this is a full and complete validation study (it was really just a curiosity thing that we thought would be interesting to pilot given she's interested in incidental memories), it does give me some confidence that the following table can be used as intended. So, I thought I would share that, just in case anyone else gets the desire to play with time lines again.
- Jeff
When dealing with witness statements we're often confronted with information of an unknown reliability. Given my interest in trying to re-create simulations of the events surrounding the different crimes, one of pieces of information I have to use are statements a witness makes about some duration. Things like when Levey and Lowende each state how long they waited after leaving the club at 1:30 due to the rain. Levey estimated 3 or 4 minutes, Lawende estimated 5. Or, when Brown estimates how long after he got home with his supper did he hear the noise in the street of the men running along Fairclough (he estimates 15 min). These intervals can be used to try and string events together.
How good that stringing together is depends upon how reliable estimates of temporal durations are. When working on the simulations, I came across a research paper where they had people estimate the duration of various events that in actuality could span very little time (under a minute) out to things that last over an hour. What they found was a tendency for people to overestimate durations under an hour and after that to tend to underestimate the duration.
What I needed, though, wasn't something that allowed me to predict how much someone would overestimate a known duration, but rather, to take an estimated duration and convert it back to a more reliable actual duration. Fortunately, the article included enough information that I was able to use their data to come up with the sort of conversion table I needed (below; basically, look up the duration the witness states in the left most column, and the best guess actual duration is the average time, although due to how rubbish people are at this, you could expect the real duration to be between the min and max durations).
While I was reasonably happy with this, I am always a bit sceptical about conversions like this until they are verified. One of my research students is looking at incidental memory formations. Basically, people are shown 120 words and they have to decide if the word is of a living or non-living thing (some words they see once, some twice, and some three times - with a minimum of 10 words in between repeitions), then they do a really boring task for 5 minutes (they watch a clock hand tick around a clock and over the course of 5 minutes, 10 times it jumps 2 numbers, like from 1 to 3 rather than from 1 to 2, and they tap a key to indicate they noticed it. Honest, it's like watching paint dry. After they do that, we show them 240 words, 1/2 of which are the words they saw initially, and they rate how confident they are they saw that particular word. From this we can get a measure of how strong their memory is, and unsurprisingly, the more often a word was repeated the stronger their memory for it was. What was suprising is the fact that people are actually pretty good at it given they had no idea they were going to be asked to recognize the words from the living/non-living task (they get over 70% correct for words they were only shown once, and it just gets better from there).
Anyway, after they do that recognition memory task, they are asked to estimate, to the nearest minute, how long the study took them. They were also asked to estimate what time it was to the nearest minute. Again, they were not aware they would be asked about these things. She's collected data from 65 people (well, 67, but one joker estimated it took 111111 minutes, and another misunderstood the question, so I've removed those two). The average estimated duration people gave was 28m 30 seconds. If I look up 28 minutes and 29 minutes as estimates in the table, the two averages are 25m 28s and 26m 30s, so one would convert an estimate of 28m 30s to 26 minutes. And the average actual duration was 26m 30s! That is pretty good in my books!
The shortest estimate given was 9 minutes (which would suggest a maximum of 24m 43s, their actual time was 26m 42s) and the longest estimate was 47 min (which would suggest a minimum of 24m 58s, and their actual duration was 26m 42s). So while our shortest estimate did fall a bit outside the 95% confidence interval, we do expect that to happen 5% of time, so that's pretty good too.
Obviously, there is no way to take a given statement and recover the exact true real duration, all one can do is go with the average and consider the range. However, as you work with more and more statements, some times the average will be too long, sometimes too short, etc, and the individual errors will tend to cancel each other out. If you have multiple people estimating the duration of the same event, you can also use probability distributions, overlap them from different estimates, and from that try and get a more reliable estimate than you can from a single person.
Given people had to book a time to do the experiment, their estimates of the actual time of day were pretty good. However, it was interesting that out of the 65 people, 3 of them thought it was an hour earlier than it was, and one thought it was an hour later! So just over 6% were still massively wrong in the time despite having come to an appointment they booked the time for!. I rather suspect, though, that under normal circumstances if we were to just randomly ask people what time they thought it was, we would see a lot more similar errors (basically, people's error for the Time of Day was related to how far off their estimate of the experiment's duration was, which is probably not too surprising).
Anyway, while it's not intended to suggest this is a full and complete validation study (it was really just a curiosity thing that we thought would be interesting to pilot given she's interested in incidental memories), it does give me some confidence that the following table can be used as intended. So, I thought I would share that, just in case anyone else gets the desire to play with time lines again.
- Jeff
Attached Files
Comment