Time Motion Studies in Healthcare: What are we talking about?

1.1 Objective

In response to the aforementioned gaps in knowledge and challenges surrounding TMS methodologies, our objective in this report is to contribute to the standardization of time motion studies by providing a disambiguation based on a broad understanding on what the concept “Time Motion Studies” currently embraces in the biomedical literature. Specifically, we aim to:

1. Review a broad sample of the biomedical literature being referenced or classified as pertaining to “time motion studies” in order to identify the current scope of the method.

2. Provide a detailed description of the distinct methods used in those articles; and

3. Present a disambiguation schema for the term “time motion studies”.

3.1 Time motion data produced by external observers

In this group, the burden of collecting the time motion data relies on an external observer. It can be performed asynchronously by having the observer analyze video recordings of the subject behavior in the work environment (also called “time-action analysis”[14][15]), but it is mostly conducted by directly observing and following the subject in real-time. Subjects might feel disturbed, and sometimes an improvement in performance can be evidenced by the presence of an external observer: a phenomenon known as the Hawthorne effect¹. Two methods were identified within this category: continuous observation and work sampling.

In continuous observation, the external observer maintains the attention on the subject and continuously records the time taken to perform one or multiple tasks, implying that the action of recording is triggered by an action performed by the subject. It is a useful approach to collect data for non-centralized tasks, sensible for short tasks and provides granular and detailed field data, but usually requires a 1:1 subject to observer ratio.

Unlike continuous observation, which measures the elapsed time for a task, work sampling identifies the task being performed at a given instant[16], repeating the measure at predefined fixed or random intervals during the observation. It relies on the repetitive nature of work and assumes the probabilistic generalization of the sampling findings to describe how workers spend their time. One of the major benefits compared to continuous observation, is that the observer can focus on multiple subjects during a single observation period. Work sampling has been reported to be efficient for studies classifying the subjects’ activities into fewer categories: with more categories describing less frequent tasks, the required number of observations increases [11], losing the advantages afforded by this method. Strictly speaking, work sampling estimates the proportion of time given to an activity based on random observations [17]. The temporality of the sampling methodology has been debated in the literature, concluding that systematic work sampling is a flawed biased estimate, and random work sampling is the recommended approach [18]. However, one of the pioneer researchers in TMS argues that the reduction in bias provided by randomization is overweighed by the complexities in scheduling the observations, advocating in favor of fixed periodic intervals[19]. We observed this issue in our review: all the studies conducting work sampling with an external observer used a systematic fixed time interval: i.e. 1 minute[20], 5 min[21]. A high frequency work sampling method was identified and referred to as “Davis observation code”[22], sampling every 15 seconds. Under optimal circumstances, work sampling has been proposed as a useful and efficient methodology for analyzing the distribution of staff work activities in relation to the types of activities they perform[23], but falls short for questions related to specific task durations, occurrences or workflow studies. A highly cited paper concludes that work sampling may not provide an acceptably precise approximation of the results that would be obtained by continuous observation time motion studies.[11]

3.2 Time motion data produced by the subject(s) being studied

In this group, the subjects provide the time-related data themselves. Although self-report can be a low cost method for measuring work activities, perceptual differences among the self-reporters can lead to discrepancies in how activities are categorized [24]. Also, subjects may either lie about what they are doing or change normal routine in order to represent data they believe are in some sense more desirable.[11] This has been proven outside TMS when comparing self-report and observation for dentists providing preventive services: self-report frequencies systematically exceeded observed frequencies[12]. Self-reports are also considered unreliable because they over-estimate contact time with patients and under-estimate the non-productive clinician time, compared to work sampling using an external observer[25]. Anecdotally, one study comparing the number of duty-hours violations among residents encountered no difference between self-reports and computerized system time-stamps[26]; however instead of reporting the agreement between the two measures, they compared if a threshold of work hours was exceeded, and not the specific durations.

Four major methods were identified in this group, which can also be classified as synchronous or asynchronous. On the asynchronous side of the spectrum we found interview/focus groups and surveys. There, the subject is asked to answer questions regarding the time it takes him to perform different steps of a process. Asynchronous self-report methods are considered limited due to the participant’s subjective evaluation of their workflow and working conditions[27], and poor estimators of durations. For example, when comparing physician recall of events duration in the operating room, surveys over-estimated the duration by 30 minutes on average, from a few minutes up to two hours, when compared to times extracted from the surgery log[28]. On the synchronous side of the spectrum, we identified active tracking and self-reported work sampling, which have been proved burdensome by nurses attempting to record activities while conducting clinical duties[24]. In active tracking, the subjects complete some kind of log with the time motion data, either after completing the task, or by the end of the work day. Self-reported work sampling involves the repeated recording of the activity taking place at pre-determined systematic or random intervals, but in this case, the subject himself is required to record the data. As previously discussed, random intervals recording are feasible and more common in this setting [29], usually using some sort of electronic device that reminds the subject in random intervals to record the data. In a comparison of self-reported work sampling and traditional work sampling for measuring nursing tasks[30], the self-report work sampling technique was found to be an unreliable method for obtaining an accurate reflection of the work tasks of ward-based nurses. Also, nurses preferred the use of an external observer. Despite that, one of the largest scale TMS corresponds to self-report work sampling by nurses in 36 hospitals[31].

3.3 Time motion data produced automatically by computerized systems

In this group, timestamps and durations of tasks are automatically generated by computerized systems when the subjects perform pre-defined tasks. Usually the movement of the subjects triggers the record of the time motion data on the systems, providing a rich “motion” dimension of the data. It is important to highlight that the name doesn’t make reference to the use of electronic tools to collect the data (such as an external observer using a tablet pc to timestamp events), but to the fact that the time stamp is being created automatically, observer-independent and without an active intention of data recording by the subject. This group includes a broad range of systems, spanning from indoor or global positioning systems, accelerometers, electrodes, radio frequency identification and diverse sensors. From the subject’s perspective, this can be considered a passive tracking method that records time motion data automatically while the subject performs their usual workflow. Examples include position-tracking devices that create time-stamps when the user approaches sensors, the time-stamping log of user-interface events on the screen of an electronic health record or sensor movements on a laparoscopic surgery training module. Although for electronic motion sensors time is a reliable variable captured[32], time-stamped logs from software usage have to be interpreted carefully. If the variable of interest is the duration of interactions with the system (i.e. charting time) it could be considered an accurate measure. However, when the variable of interest is deduced from the automated EHR timestamp as a proxy (i.e. time of patient transfer to another unit) it might not be an accurate measure. For example, a TMS conducted in the emergency department compared continuous observation and time extracted from EHR timestamps, concluding that durations extracted from the EHR were recorded 2 minutes before they were actually observed (median, interquartile range 31 min before to 3 min after)[13].

3.4 Special case: Logged time on charts/documents/databases

A large portion of the articles (21%) conducted the time analysis on logged time from databases, chart reviews or existing documents. Although this could represent a standalone group of studies, it actually embraces two possible means of time data capture: a first scenario where the subject actively records a time or a duration in the system/record, and a second scenario where the logged time data is being automatically recorded in the system by other actions performed by the user (the user does not actively input time data on to the record). Thus, the first scenario corresponds to a subset of the active tracking self-report group, while the second scenario corresponds to a subset of the UI timestamps from the computerized systems group. Due to the previously presented evidence that emphasizes the subjectivity of the self-reported data, this distinction is not minor. Although we could assume that most of the data in this group came from active tracking self-report, this wasn’t systematically reported. Thus, we decided to represent these studies in a standalone category, while encouraging future researchers to report the data source of their logs and databases. An interesting discussion arises from the assumption that this group represents a distinct scenario from self-report active tracking: a scenario where the action of self-registering times/ durations is actually part of the routine workflow of the subjects. However, the transition from being a prolonged self-report study to collecting time data as part of the routine workflow is unclear and the effect on data accuracy has not been studied.

3.5 Other studies

Seven articles returned by our query did not collect or analyze any task/event time duration. The articles corresponded to two time-series studies[33][34] (where time is an independent variable), an interview to assess why a delay existed[35], two work-load calculation models for the emergency department[36][37], a qualitative participant observation with no quantitative time motion data captured[38], and a survey to validate time-related vocabulary[39]. One article did not provide any details on the method used to collect the data, and was left unclassified. An interesting highlight is that these seven studies were classified as TMS by MeSH, but not by the authors (not mentioned in the title or the abstract).

3.6 MeSH vs. authors: discrepancies

We iteratively combined Boolean operators in our query to identify articles retrievable by the “Time Motion Studies” [Majr], “time motion study” or “time and motion study” string on the title/abstract, or both, and compared the number of results. By using MeSH [Majr], 216 articles were returned, compared to 99 returned by the string on the title/abstract. At first, we thought this might be due to a difference in sensitivity (one being a subset of the other), but when we studied the intersections and union of the groups, a different set of articles were retrieved [Figure 1]: MeSH and the authors seem to have a different understanding of time motion studies. We compared the relative prevalence of the different identified methods [Figure 2]. Authors mentioning time motion study in the title/abstract mostly refer to methods using an external observer, while MeSH noticeably extends the scope in databases/chart reviews and in asynchronous self-report methods (surveys/interviews). Continuous observation represented the 32.4% of the TMS articles indexed by mesh and 77.1% of the self-reported TMS articles.

4.1 Time Motion Studies: what are we talking about?

The original use of the term time motion study (combining the work of Taylor and the Gilbreths) refers to a method for improving efficiency and establishing employee productivity standards in which a task is broken into steps, the sequence of movements performed by the subject to accomplish those steps is observed to detect redundant motion, and precise time taken for each movement is measured. However, our query results returned a broad spectrum of distinct methodologies referred to as “time motion studies”. They ranged from surveys, patient records reviews, self-reports and work-sampling. There is an evident misunderstanding and contradictory definitions that need to be clarified.

Based on our results, we present a disambiguation schema. Currently “time motion studies” is used to refer to:

1. The conglomeration of studies using a broad spectrum of dissimilar methods whose only common factor is the capture and/or analysis of the time required to complete one or more events². An important fact to highlight is that this “least common denominator” leans towards the time study portion of time motions studies. Although most automated timestamps consider motion at a very granular level, data produced by external observers only sometimes considers motion, and usually at a high level by recording the subject location. Self-reported data rarely considers motion. This emphasis on the time study side of time motion studies can also be evidenced when assessing the entry terms for “Time and Motion Studies” in the MeSH hierarchy: “Time Studies”, “Studies, Time”, “Time Study” and “Study, Time”[40].

2. The use of an external observer capturing data continuously (no work-sampling). Although not unanimous, the majority (77.1%) of the articles containing “time motion study” in the title/abstract referred to the use of an external observer doing continuous data capture, which makes sense historically since it correlates more to the original 1900s’ time motion method.

We believe that it would be more scientifically and historically accurate to use “time motion studies” to refer to the uses of external observer capturing data continuously. However, the fact that it only represents 32.4% of the “Time motion studies” [Majr] indexed articles, the broad use of this conception and the influence of work sampling supporters, present as an impediment for trying to rectify the definition. Instead, we believe that maintaining compliance with the expanded conception and establishing the clear specification “continuous³ observation time motion studies” will allow a better understanding. This premise, accompanied by the present review of methods embraced by TMS will allow future researchers to properly identify and refer to a desired method. We encourage TMS researchers to thoroughly describe their methodologies and identify the method being used [Figure 3]. If the method selected is continuous observation TMS, we suggest to follow the checklist proposed by Zheng[7] to describe the details of the implementation of the method.

4.3 Expanding Continuous Observation Time Motion Studies and future work in TMS standardization

Continuous observation time motion studies represented the majority (42.2%) of the articles classified as time motion studies in our 10 year-span review. When analyzing the trend of methods over time, despite a year to year variability in the distribution, a clear trend towards continuous observation was observed: 35% of the methods used in 2003–2007 corresponded to continuous observation, compared to 48% in 2008–2012. This supports the theory of continuous observation being considered the gold standard methods: the research community accepts them as the method of preference to collect time/duration data in healthcare. Thus, standardization efforts should keep focusing on this method. Based on our findings, two clarifications will further contribute to the common understanding of TMS methodologies, specifically continuous observation TMS.

First, among methods based on external observers, the distinction between work sampling and continuous observation is not universally understood. For example, one article using continuous observation[41] provided evidence as to why the method used was superior (participant observation of time allocation with continuous task duration recording) by citing an article that examined work-sampling[25]. The distinction between triggering the recording by an action from the subject (continuous observation) versus triggering the recording of data by a time interval (work sampling) gets even more confusing when variation of the methods are poorly described. For example, some studies using continuous observation with paper-based tools record the start and stop times of observed tasks with a detail of minutes, not seconds (easier for the observer, but produces a loss of detail and accuracy of the observation, thus losing the ability to track tasks fragmentation and quick task switching). This scenario can be easily confused with a systematic one-minute interval work sampling approach, where the observer records the current activity every elapsed minute. A rare, more complex sub-type of work sampling is particularly confusing: a one minute interval work sampling approach where the observer, instead of recording the current activity, recorded the activity that most represented the physician’s action during the previous minute[42]. We suggest naming this “task predominance work sampling”.

Second, we believe that a more granular description of continuous observation TMS will help distinguish sub techniques, which in turn might explain some of the variability previously described. Within continuous observation, we encountered three distinct data schemas naming them “single duration measurement”, “milestones timing” and “workflow time study”, corresponding to 19.7%, 18.8% and 61.5% of continuous observation TMS respectively.

In single duration measurement, the external observer measures the duration of a single event, i.e. the observer only records the subjects’ time spent in the bathroom[43] or the time required to walk 50 feet[44]. Zheng et al. differentiated this group from the rest of continuous observation methods, excluding it from his review and referring to it as “efficiency studies on isolated events, such as workaround times”. We believe that a further specification can be declared within the remaining group, identifying two distinct sub-techniques. In one approach, researchers interpret and model qualitative data onto theoretical activity diagrams a-priori[45], and then enrich the abstracted workflow with time-data[46] by having observers record the duration of expected ordered milestones[47]. We named this approach “milestones timing”, which is the most similar to the original method described by Taylor. Examples include measuring six vaccination steps[48], timing the duration of five stages in a esophageal manometry[49], or developing an idealized medication administration flowchart and time the duration of each predicted step[50],[51]. While “single duration measurement” and “milestones timing” provide a relatively simple framework for the observers, it has been broadly reported that the nature of clinical work is unpredictable[52][53], with an unusual number of imponderables, nonstandard environments and equipment, caring for distinct variable patients. This non-linear and interruptive nature of clinical work encourages researchers to use a more granular approach[54] provided by the third sub-technique identified, which we named “workflow time study”. In this more comprehensive schema, the observer records the occurrence and duration of unpredicted instances of tasks, producing a data schema of time-stamped tasks, which accounts for task fragmentation, interruptions and work variability. This proposed differentiation is not trivial, since it might partially explain the methodological variability previously reported in continuous observation TMS. For example, the number of tasks being observed will certainly be lower for milestones timing than workflow time studies, or the hours of training will significantly differ if the observer has to focus on a single event duration v/s identifying and timing a set of tasks. Also, humans have been proven to be good time keepers in single duration measurements compared to electronic sensors[55], and reliable for tracking simple movement patterns compared to GPS[56]. However, for more complex scenarios like workflow time studies, reliability becomes a major concern. Assessing inter-observer reliability for each sub-technique will differ: single duration measurement and milestones timing will focus on the duration of the pre-specified interval(s), while workflow time studies require a multidimensional inter-observer reliability assessments to compare the naming, count and duration of the tasks identified. Differences are also expected for the duration of the observations and the unit of observation, etc…

We believe this detailed description will help better understand the variability among these methods. With a common understanding, future efforts on standardizing continuous observation TMS can be aligned and potentiate collaborations in pending issues such as how to record multi-tasking accurately, best approaches for recording interruptions, a proper inter-observer reliability assessment and tasks definitions concerns.

4.4 Limitations

The use of a single database (PubMed) might have limited the scope of the articles retrieved and only represents the indexing trend of MEDLINE, which might differ with other biomedical bibliographic databases, or business/management literature. Also, these findings only reflect time motion studies from the biomedical domain, specifically healthcare, therefore these findings might not apply to other fields concerning these studies (i.e. industrial engineering).