# Community Guidelines Enforcement Report (CGER) - Dashboard Generator Toolkit

## 1. Introduction
This toolkit contains the necessary components to generate interactive, web-based dashboards from the raw CSV data of TikTok's Community Guidelines Enforcement Report. The pipeline leverages a Python script to parse flat-format data and dynamically inject it into styled HTML templates, producing standalone files ready for external publication.

---

## 2. Understanding the Data Structure (CSV)
The raw data is provided in a flat, long-format CSV file. Each row represents a single pre-aggregated data point. The data structure is designed for flexibility, allowing new metrics and time periods to be added seamlessly. 

The CSV consists of four key components:

### A. Metrics
* **`Metric`**: The specific measurement being reported (e.g., 'Total videos removed', 'Proactive removal rate', 'Category share'). 
* **Note**: Always filter analysis by a specific metric. There is no 'All' value in this column.

### B. Time Period
* **`Period type`**: The frequency categorization (e.g., 'Quarterly').
* **`Period`**: The specific time window (e.g., 'Oct-Dec 2025'). 
* **Note**: The Python script automatically sorts periods chronologically (Q1 -> Q4) to ensure accurate rendering on timeline charts. Dashboard 4 (Policies) is specifically restricted to Q4 2025 and later due to categorization changes.

### C. Granularities
These columns provide deeper breakdowns of the metrics. When doing top-level analysis, look for the 'All' or 'Global' values in these fields.
* **`Policy type` & `Issue`**: Details the specific community guideline (e.g., Policy: 'Safety and Civility', Sub-policy: 'Harassment and Bullying').
* **`Task type` & `Task`**: Details the moderation system used (e.g., 'Automation' vs. 'Human moderation').
* **`Location` & `Market`**: Details the geographic region (e.g., 'Global', 'United States', 'France').

### D. Results
* **`Result`**: The numerical value for the row. This can be a raw count (e.g., 1,500,000) or a percentage/rate (e.g., 0.952 representing 95.2%). The Python script automatically handles conversion based on the metric type. Values may be omitted for privacy if they represent fewer than 1,000 users.

---

## 3. The Dashboard Generation Pipeline
The generation process relies on two main components:

1.  **HTML Templates (`templates/` folder)**: 
    * These are skeleton HTML files (e.g., `1_Prevalence_Template.html`) containing the styling (CSS), layout, and chart rendering logic (Chart.js / Plotly).
    * They contain a specific injection marker (`/* INJECT_DATA */`) where the Python script will insert the JSON data.
    
2.  **The Python Engine (`build_dashboards.py`)**:
    * The script reads the raw CSV and filters for 'Global' or 'All' market data (except for Dashboard 5, which handles specific country/language granularities).
    * It extracts and formats the required metrics for each specific dashboard into a JSON payload.
    * It opens the corresponding HTML template, injects the JSON data via Regex (bypassing Unicode escaping issues), applies dynamic JavaScript patches (like X-Axis formatting for years and Y-Axis volume suffixes), and saves a new, fully populated HTML file.

---

## 4. Usage Instructions

### Prerequisites
* Python 3.x installed.
* `pandas` library installed (`pip install pandas`).
* Ensure your directory contains the `build_external_cger_dashboards.py` script and a `templates/` subfolder with your master HTML templates.

### Execution Steps
1.  **Place the Data**: Ensure the latest raw data CSV file is in the same directory as the script. The script uses `glob` to automatically detect the CSV, so the exact filename does not matter as long as it is the only CSV in the root folder.
2.  **Run the Script**: Execute the Python script from your terminal/command prompt:
    `python build_external_cger_dashboards.py`
3.  **Retrieve Output**: The script will create a new folder named `external_dashboards/`. Inside, you will find the generated, standalone HTML dashboards (e.g., `1_Prevalence.html`, `2_Volume.html`), ready for review and deployment.
