Longitudinal Implementation Strategy Tracking System (LISTS)

A man and woman sit at a desk looking at a computer with a whiteboard behind them that has data points written on it.

Longitudinal Implementation Tracking System (LISTS)

What Is LISTS?

LISTS offers a systematic method for documenting, specifying, and tracking the use of implementation strategies in a given project or study. It was developed collaboratively between researchers and practitioners to fill a research gap in systematically tracking implementation strategy use and modifications in research studies and other implementation efforts.

By accurately characterizing the dynamic use of implementation strategies and how they change over time, LISTS increases the precision and reliability of changes in implementation strategy use, and encourages the use of common data elements to support data analysis across sites and synthesis across studies. It includes three components: 1) a strategy assessment, 2) a data capture platform, and 3) a user guide.

LISTS enables users to bundle strategies together and visualize strategies through an interactive dashboard, and incorporates timeline follow-back procedures to support accurate, ongoing tracking of implementation strategies.

Launch the LISTS online tool

Check out the instructions and resources below to learn what data elements are needed to use the tool.

Why Use LISTS?

- Data Collection: LISTS allows for the systematic collection of data at multiple points throughout the implementation process. This helps capture progress and changes over time.
- Analysis Integration: By providing a structured framework, LISTS enables the integration of various data points into the user’s analysis. This deepens understanding of the effectiveness of different strategies and interventions.
- Outcome Assessment: With the data collected, LISTS facilitates the assessment of implementation outcomes. It helps by identifying patterns, trends, and any deviations from expected results.
- Informed Decision-Making: The insights gained from LISTS can guide decision-making. It allows users to determine whether to continue, modify, or halt a particular strategy by tracking the strategy’s performance.
- Alignment with Goals: LISTS ensures that the strategies being implemented are aligned with the overall goals and objectives. It provides a measurable way to track progress and make necessary adjustments.

Tools and Procedures to Get Started

Users are encouraged to use the companion spreadsheet below to specify the data elements of each strategy before using the LISTS interactive online tool. Users will also need to sign up and create an account to use the tool. Accounts are free and easy to create.

Download the step-by-step user guide

View the interactive user guide

Download the companion spreadsheet

Recommended Procedures

1. Identify your team and who will be responsible for LISTS tracking and updating. These individuals should have direct knowledge of implementation activities. Example team members include:

- Implementation scientist(s)
- Investigators and co-investigators
- Embedded physician scientists
- Project/implementation coordinator(s)
- Practice champions

2. Meet with your team at routine intervals

- Weekly/monthly/quarterly depending on the nature of the study, centrality of modification/adaption, study design, phase of the study

3. Initially populate LISTS

- Review the full list of ERIC discrete strategies (prompt)
- Enter strategies into the Excel spreadsheet
- Confirm LISTS elements (other team members, calendars, meeting notes)
- Establish a point person for compiling/entering strategies into the online tool

4. Update LISTS using the timeline follow-back procedure

- Set up regular check-ins with implementers and LISTS team members re: changes/new strategies/discontinuations
- Review entered strategies to assess for changes routinely

Necessary Data Elements

Data elements needed before using the LISTS online tool:

- Key study dates (e.g., grant start, project start, implementation go-live, other)
- Implementation strategy name(s) and definition(s)
- Strategy specifications
  - - Actors
    - Action
    - Action target(s)
    - Dose

- Dates when the strategy starts/stops
- Implementation outcomes each strategy is targeting

Data elements recommended before using the LISTS online tool:

- Implementation determinants the strategy is addressing (optional)
  - - The Consolidated Framework for Implementation Research (CFIR)
    - Exploration, Preparation, Implementation, Sustainment (EPIS) Framework
    - Practical, Robust Implementation and Sustainability Model (PRISM)
    - Socio-Ecological Model (SEM)

LISTS Presentations

Using LISTS to Improve the Rigor of Implementation Research

Seminar presented on June 3, 2021, by J.D. Smith, PhD.

Watch this seminar recording to learn more about LISTS as a feasible approach for longitudinal tracking, and how it can facilitate synthesis across units and within projects and cross-project initiatives of similar interventions.

More on the Development of LISTS

An iterative process between implementation researchers and practitioners was used to develop, pilot test, and refine LISTS prior to evaluating its use in three trials. The LISTS method was used with research and practice teams for approximately 12 months and subsequently evaluated for its feasibility, acceptability, and usability using established instruments and novel questions developed specifically for the study. Initial evaluation of LISTS indicates that it is a feasible and acceptable method, with content validity, for characterizing and tracking the use of implementation strategies over time. The software was developed by the Research Application Design and Development team at Northwestern University.

LISTS aligns with established taxonomies for specifying and justifying implementation strategies, while simultaneously offering flexibility for users by enabling them to link their strategies to one of several determinant frameworks as well as a wide variety of implementation and service outcomes. Informed by the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS), LISTS also gives users the ability to track adaptations and modifications made to strategies over time. Potential future developments for LISTS include additional features, fewer data elements, and interoperability with alternative data entry platforms.

The development of LISTS was made possible by funding from NIH. ISCI is funded by an administrative supplement to the Third Coast Center for AIDS Research, an NIH-funded center (P30 AI117943).

Suggested LISTS Citation

Smith JD, Norton WE, Mitchell SA, Cronin C, Hassett MJ, Ridgeway JL, Garcia SF, Osarogiagbon RU, Dizon DS, Austin JD, Battestilli W, Richardson JE, Tesch NK, Cella D, Cheville AL, DiMartino LD, the IMPACT Consortium. The Longitudinal Implementation Strategy Tracking System (LISTS): Feasibility, Usability, and Pilot Testing of a Novel Method. Implementation Science Communications, 4:153, November 2023.