Skip to content

Toolkits

Each toolkit groups the tools of one stage of a longitudinal analysis. They chain as follows, although each one can also be used on its own, since every tool reads and writes files on disk (CSV for most tools, with dedicated converters for Parquet and Stata).

flowchart LR
    A["Data Preparation<br/>(Stage A)"] --> B["Data Preprocessing<br/>(Stage B)"]
    B --> C["Benchmarking and Modelling<br/>(Stage C)"]
    C -- "saved models" --> D["Explainability<br/>(Stage D)"]
Stage A Data Preparation Generation of synthetic longitudinal cohorts with known trends, shocks or missingness, and conversion of files between CSV, Parquet and Stata. TrendPatternsMissingDataScenariosStataToCsv Stage B Data Preprocessing Cleaning, imputation, category harmonisation, and long to wide (or cross-sectional) reshaping of repeated measures. CleanDatasetMissingImputationPivotLongToWide
Stage C ยท Main toolkit Benchmarking and Modelling Ranks standard and longitudinal estimators all against all, under one shared protocol, so that the value of machine learning over the statistical analysis in place is measured on the data at hand; then trains one chosen estimator in depth. BenchStandardAndLongitudinalMLBenchmarkStandardMLBenchmarkLongitudinalMLStandardMachineLearningLongitudinalMachineLearning
benchmark_report.txt
1. Logistic Regression                    0.796
2. MerWavTimePlus + Lexico Random Forest  0.779
3. Random Forest                          0.759
4. SepWav (Voting) + Random Forest        0.722
5. AggrFunc (Mean) + Logistic Regression  0.605
Stage D Explainability Explains the predictions of a saved model, e.g., the best estimator of a benchmark, with SHAP values: which features, and which waves, drive them. SHAPAnalysis
shap_analysis_report.txt
Top features by mean absolute SHAP value
1. mood_score_w3: 0.165802
2. mood_score_w2: 0.076778
3. sleep_score_w3: 0.042331
4. mood_score_w1: 0.040893

Proposing a New Toolkit or Tool

A new tool (e.g., an imputer, an estimator or a file converter) usually fits an existing toolkit, whereas a new toolkit is warranted when a whole stage of longitudinal analysis is missing. Either way, the starting point is a feature request on GitHub.

Request a data preparation tool Request a preprocessing tool Request a benchmarking or modelling tool Request an explainability tool