tt_viz.qmd

---
title: "TikTok Coordinated Sharing Network"
author: "Fabio Giglietto"
format: html
date: "`r Sys.Date()`"
params:
  load_from_snapshot: FALSE
---

# Introduction

This document serves as a proof of concept, developed within the framework of the [Vera.ai Horizon EU project](https://www.veraai.eu/). It presents a comprehensive methodology for tracking and analyzing coordinated sharing activities on TikTok, employing the [`traktok`](https://github.com/JBGruber/traktok) package for data collection via the TikTok Research API and [`CooRTweet`](https://github.com/nicolarighetti/CooRTweet) for the analysis of coordinated behavior patterns.

**Initial Discovery:** The analysis initiates by focusing on content tagged with the #moskow hashtag. This first step successfully identifies a preliminary group of accounts involved in coordinated sharing activities in the aftermath of the attack in Moscow. This discovery lays the groundwork for an extensive examination of coordinated dynamics across social media platforms.

On April 19, 2024, we received a list of 513 problematic accounts from a trusted partner within the Vera AI consortium. Accounts that were mentioned at least twice on this list have been added to our pool of monitored accounts.

**Daily Monitoring and Analysis:** Subsequent to the initial identification, the methodology transitions into a phase of daily monitoring. In this phase, the script consistently retrieves videos posted by the previously identified accounts, with the goal of detecting both ongoing and emerging instances of coordinated behavior. As new accounts manifesting coordinated behavior (time_window = 180, min_participation = 2, edge_weight = 0.5) are discovered, they are incorporated into the daily monitoring routine.

This approach ensures continuous updates on the number of newly discovered coordinated accounts, highlighting the fluid nature of social media coordination. Enhanced by interactive visualizations, the analysis sheds light on the shifting landscape of account activities and the intricate network of interactions among them on the TikTok platform.

By delineating these processes, the proof of concept underscores the potential for advanced analytical tools to reveal and understand the complex phenomena of coordinated social media behavior within the context of significant societal events.

```{r setup, include=FALSE, message=FALSE}
setwd("/home/mine/VERAAI_WP4/")

# Library loading and initial setup
library(tidyverse)
library(CooRTweet)
library(httr)
library(purrr)
library(scales)
library(stringr)
library(igraph)
library(visNetwork)
library(ggplot2)
library(jsonlite)
library(lubridate)
library(progress)
library(logger)

knitr::opts_chunk$set(echo = FALSE)

# Initialize logging
log_info("Starting TikTok coordinated activity detection script")

# At the beginning of your script, add:
global_error_list <- list()

# Secure credential retrieval
get_tiktok_credentials <- function() {
  list(
    client_key = Sys.getenv("TIKTOK_CLIENT_KEY"),
    client_secret = Sys.getenv("TIKTOK_CLIENT_SECRET")
  )
}

# Function to request an access token
request_access_token <- function(credentials) {
  tryCatch({
    url <- "https://open.tiktokapis.com/v2/oauth/token/"
    headers <- c('Content-Type' = 'application/x-www-form-urlencoded')
    body <- list(
      client_key = credentials$client_key,
      client_secret = credentials$client_secret,
      grant_type = "client_credentials"
    )
    
    response <- httr::POST(url, add_headers(.headers = headers), body = body, encode = "form")
    
    if (httr::status_code(response) == 200) {
      content <- httr::content(response, "parsed")
      log_info("Successfully obtained access token")
      return(content$access_token)
    } else {
      log_error(paste("Failed to retrieve access token. Status code:", httr::status_code(response)))
      stop(paste("Failed to retrieve access token. Status code:", httr::status_code(response)))
    }
  }, error = function(e) {
    log_error(paste("Error in request_access_token:", e$message))
    stop(e)
  })
}

# Main execution
main <- function() {
  credentials <- get_tiktok_credentials()
  
  if (!params$load_from_snapshot) {
    # Ensure environment variables are set
    if (is.na(Sys.getenv("TIKTOK_CLIENT_KEY")) | is.na(Sys.getenv("TIKTOK_CLIENT_SECRET"))) {
      log_error("Environment variables for TikTok API not set")
      stop("Environment variables for TikTok API not set. Please set TIKTOK_CLIENT_KEY and TIKTOK_CLIENT_SECRET.")
    }
    
    tryCatch({
      access_token <- request_access_token(credentials)
    }, error = function(e) {
      log_error(paste("Authentication with TikTok API failed:", e$message))
      stop("Authentication with TikTok API failed. Error message: ", e$message)
    })
    
  } else {
    # Load data from the latest snapshot
    all_videos <- readr::read_csv("/home/mine/VERAAI_WP4/output/latest_snapshot.csv",
                                  col_types = cols(video_id = col_character(), 
                                                   effect_ids = col_character(), music_id = col_character(), 
                                                   hashtag_names = col_character()))
    recent_videos <- NA
  }
  
  # Try reading the CSV file and handle possible errors
  tryCatch({
    account_ids <- readr::read_csv("./lists/tiktok_coordinated_accounts_ids.csv")
  }, error = function(e) {
    log_error(paste("Failed to read TikTok coordinated accounts ID CSV:", e$message))
    stop("Failed to read TikTok coordinated accounts ID CSV. Error: ", e$message)
  })
  
  coord_users <- unique(account_ids$x)
  enddate <- Sys.Date()
  startdate <- enddate - 15
  
  # Return the values we want to be globally available
  list(
    coord_users = coord_users,
    enddate = enddate,
    startdate = startdate,
    access_token = if(exists("access_token")) access_token else NULL,
    all_videos = if(exists("all_videos")) all_videos else NULL,
    recent_videos = if(exists("recent_videos")) recent_videos else NULL
  )
}

# Run main and assign results to global environment
result <- main()
list2env(result, .GlobalEnv)

```

```{r fetch_recent_videos, message=FALSE, eval=!params$load_from_snapshot}

# Helper function to generate 12-hour time chunks
generate_time_chunks <- function(start_date, end_date) {
  start_time <- as.POSIXct(start_date)
  end_time <- as.POSIXct(end_date)
  seq(start_time, end_time, by = "12 hours")
}

get_recent_videos <- function(usernames, start_time, end_time, access_token, max_retries = 3) {
    attempts <- 0
    all_results <- list()
    pagination_attempts <- 0
    
    # Convert times to required format (YYYYMMDD)
    formatted_start <- format(start_time, "%Y%m%d")
    formatted_end <- format(end_time, "%Y%m%d")
    
    url <- "https://open.tiktokapis.com/v2/research/video/query/?fields=id,create_time,region_code,share_count,view_count,like_count,comment_count,video_description,username"
    
    query_body <- list(
        query = list(
            and = list(
                list(
                    operation = "IN",
                    field_name = "username",
                    field_values = as.list(usernames)
                )
            )
        ),
        start_date = formatted_start,
        end_date = formatted_end,
        max_count = 100  # Adjust as needed
    )
    
    repeat {
        attempts <- attempts + 1
        log_info(paste("Starting attempt", attempts, "for time chunk", formatted_start, "to", formatted_end))
        
        tryCatch({
            Sys.sleep(5)  # Sleep for 5 seconds between attempts
            
            # Convert the query body to JSON
            query_json <- jsonlite::toJSON(query_body, auto_unbox = TRUE)
            
            # Make the API request
            response <- httr::POST(
                url = url,
                httr::add_headers(
                    Authorization = paste("Bearer", access_token),
                    "Content-Type" = "application/json"
                ),
                body = query_json
            )
            
            # Check the response
            if (httr::status_code(response) == 200) {
                content <- httr::content(response, "parsed")
                
                if (length(content$data$videos) == 0) {
                    log_warn(paste("No videos found for time chunk", formatted_start, "to", formatted_end, "on attempt", attempts))
                    global_error_list <<- c(global_error_list, list(list(
                        type = "api_response",
                        context = "video_retrieval",
                        start_time = formatted_start,
                        end_time = formatted_end,
                        attempt = attempts,
                        message = "No videos found in API response"
                    )))
                    return(data.frame())
                }
                
                # Process the response into a data frame
                result <- do.call(rbind, lapply(content$data$videos, function(video) {
                    data.frame(
                        id = ifelse(is.null(video$id), NA, video$id),
                        create_time = ifelse(is.null(video$create_time), NA, as.POSIXct(video$create_time, origin = "1970-01-01")),
                        region_code = ifelse(is.null(video$region_code), NA, video$region_code),
                        share_count = ifelse(is.null(video$share_count), NA, video$share_count),
                        view_count = ifelse(is.null(video$view_count), NA, video$view_count),
                        like_count = ifelse(is.null(video$like_count), NA, video$like_count),
                        comment_count = ifelse(is.null(video$comment_count), NA, video$comment_count),
                        video_description = ifelse(is.null(video$video_description), NA, video$video_description),
                        username = ifelse(is.null(video$username), NA, video$username),
                        stringsAsFactors = FALSE
                    )
                }))
                
                all_results[[length(all_results) + 1]] <- result
                
                log_info(paste("Retrieved", nrow(result), "videos for time chunk", formatted_start, "to", formatted_end, "on attempt", attempts))
                
                # Check if there are more pages
                if (!is.null(content$data$has_more) && content$data$has_more) {
                    pagination_attempts <- pagination_attempts + 1
                    query_body$cursor <- content$data$cursor
                    query_body$search_id <- content$data$search_id
                    log_info(paste("Pagination: Moving to page", pagination_attempts + 1, "for time chunk", formatted_start, "to", formatted_end))
                } else {
                    break  # Exit the loop if no more pages
                }
            } else {
                error_content <- httr::content(response, "parsed")
                error_message <- paste("API request failed with status code:", httr::status_code(response), 
                                       "Error message:", ifelse(is.null(error_content$error), "Unknown error", error_content$error))
                log_warn(paste(error_message, "for time chunk", formatted_start, "to", formatted_end, "on attempt", attempts))
                global_error_list <<- c(global_error_list, list(list(
                    type = "api_error",
                    context = ifelse(pagination_attempts > 0, "pagination", "initial_request"),
                    start_time = formatted_start,
                    end_time = formatted_end,
                    attempt = attempts,
                    pagination_attempt = pagination_attempts,
                    status_code = httr::status_code(response),
                    error_message = error_message
                )))
                # Return any results we've gathered so far, even if incomplete
                if (length(all_results) > 0) {
                    return(do.call(rbind, all_results))
                } else {
                    return(data.frame())
                }
            }
        }, error = function(e) {
            log_warn(paste("An unexpected error occurred for time chunk", formatted_start, "to", formatted_end, "on attempt", attempts, ":", e$message))
            global_error_list <<- c(global_error_list, list(list(
                type = "unexpected_error",
                context = ifelse(pagination_attempts > 0, "pagination", "initial_request"),
                start_time = formatted_start,
                end_time = formatted_end,
                attempt = attempts,
                pagination_attempt = pagination_attempts,
                error_message = e$message
            )))
            # Return any results we've gathered so far, even if incomplete
            if (length(all_results) > 0) {
                return(do.call(rbind, all_results))
            } else {
                return(data.frame())
            }
        })
        
        if (attempts >= max_retries) {
            log_warn(paste("Max retries exceeded for time chunk", formatted_start, "to", formatted_end, "after", max_retries, "attempts"))
            global_error_list <<- c(global_error_list, list(list(
                type = "max_retries_exceeded",
                start_time = formatted_start,
                end_time = formatted_end,
                max_retries = max_retries
            )))
            break
        }
    }
    
    if (length(all_results) > 0) {
        final_result <- do.call(rbind, all_results)
        return(final_result)
    } else {
        return(data.frame())
    }
}

# Main processing loop
time_chunks <- generate_time_chunks(startdate, enddate)
failure_count <- 0
incomplete_count <- 0
total_chunks <- length(time_chunks) - 1

tryCatch({
    pb <- progress_bar$new(format = "[:bar] :percent :etas", total = total_chunks, clear = FALSE)
    
    recent_videos_list <- map2(head(time_chunks, -1), tail(time_chunks, -1), function(start_time, end_time) {
        tryCatch({
            videos <- get_recent_videos(coord_users, start_time, end_time, access_token = access_token, max_retries = 10)
            
            if (nrow(videos) == 0) {
                failure_count <<- failure_count + 1
                log_warn(paste("Failed to retrieve videos for time chunk", start_time, "to", end_time))
                global_error_list <<- c(global_error_list, list(list(
                    type = "chunk_retrieval_failure",
                    start_time = start_time,
                    end_time = end_time,
                    message = "No videos retrieved"
                )))
            } else {
                # Check if the results might be incomplete
                time_diff <- as.numeric(difftime(end_time, start_time, units = "hours"))
                if (time_diff > 12 || nrow(videos) == 100) {  # Assuming max_count is 100
                    incomplete_count <<- incomplete_count + 1
                    log_warn(paste("Potentially incomplete results for time chunk", start_time, "to", end_time))
                    global_error_list <<- c(global_error_list, list(list(
                        type = "incomplete_results",
                        start_time = start_time,
                        end_time = end_time,
                        video_count = nrow(videos),
                        message = "Results may be incomplete"
                    )))
                }
            }
            pb$tick()
            return(videos)
        }, error = function(e) {
            failure_count <<- failure_count + 1
            log_warn(paste("Error in processing time chunk", start_time, "to", end_time, ":", e$message))
            global_error_list <<- c(global_error_list, list(list(
                type = "chunk_processing_error",
                start_time = start_time,
                end_time = end_time,
                message = e$message
            )))
            pb$tick()
            return(data.frame())
        })
    })
    
    recent_videos_list <- compact(recent_videos_list)  # Remove empty data frames
    
    if (length(recent_videos_list) == 0) {
        log_warn("All API queries failed or returned no data. Proceeding with empty recent_videos_list.")
        recent_videos <- data.frame()
        global_error_list <<- c(global_error_list, list(list(
            type = "all_queries_failed",
            message = "All API queries failed or returned no data"
        )))
    } else {
        recent_videos <- bind_rows(recent_videos_list)
    }
    
}, error = function(e) {
    log_error(paste("Error in fetching recent videos:", e$message))
    log_error(paste("Stack trace:", paste(sys.calls(), collapse = "\n")))
    global_error_list <<- c(global_error_list, list(list(
        type = "overall_process_error",
        message = e$message,
        stack_trace = paste(sys.calls(), collapse = "\n")
    )))
    recent_videos <- data.frame()
})

log_info(paste("Number of API failures:", failure_count, "out of", total_chunks, "time chunks"))
log_info(paste("Number of potentially incomplete results:", incomplete_count, "out of", total_chunks, "time chunks"))
api_failure_summary <- list(failures = failure_count, incomplete = incomplete_count, total_chunks = total_chunks)

# Add a summary of errors to global_error_list
global_error_list <<- c(global_error_list, list(list(
    type = "api_failure_summary",
    failures = failure_count,
    incomplete = incomplete_count,
    total_chunks = total_chunks
)))


```

```{r fetch_all_videos, message=FALSE, eval=!params$load_from_snapshot}

# Check if recent_videos exists and has the expected structure
if (!exists("recent_videos") || !is.data.frame(recent_videos) || nrow(recent_videos) == 0) {
  log_warn("recent_videos is not defined, not a data frame, or empty. Creating an empty unique_videos_desc.")
  unique_videos_desc <- data.frame(video_description = character(0))
} else {
  # Check if video_description column exists
  if (!"video_description" %in% names(recent_videos)) {
    log_error("video_description column not found in recent_videos.")
    global_error_list <<- c(global_error_list, list(list(type = "data_structure", message = "video_description column not found in recent_videos")))
    unique_videos_desc <- data.frame(video_description = character(0))
  } else {
    # Create unique_videos_desc
    unique_videos_desc <- recent_videos %>%
      mutate(video_description = tolower(video_description)) %>%
      group_by(video_description) %>%
      filter(n() >= 2) %>%
      ungroup() %>%
      select(video_description) %>%
      filter(video_description != "" & nchar(video_description) >= 80) %>%
      distinct()
    
    log_info(paste("Created unique_videos_desc with", nrow(unique_videos_desc), "unique descriptions"))
  }
}

# Custom function to fetch videos by description
get_videos_by_description <- function(desc, startdate, enddate, access_token, max_retries = 3) {
    attempts <- 0
    all_results <- list()
    
    start_date <- format(as.Date(startdate), "%Y%m%d")
    end_date <- format(as.Date(enddate), "%Y%m%d")
    
    url <- "https://open.tiktokapis.com/v2/research/video/query/?fields=id,create_time,region_code,share_count,view_count,like_count,comment_count,video_description,username"
    
    query_body <- list(
        query = list(
            and = list(
                list(
                    operation = "EQ",
                    field_name = "keyword",
                    field_values = list(desc)
                )
            )
        ),
        start_date = start_date,
        end_date = end_date,
        max_count = 100
    )
    
    repeat {
        attempts <- attempts + 1
        log_info(paste("Starting attempt", attempts, "for description:", desc))
        
        tryCatch({
            Sys.sleep(5)  # Sleep for 5 seconds between attempts
            
            query_json <- jsonlite::toJSON(query_body, auto_unbox = TRUE)
            
            response <- httr::POST(
                url = url,
                httr::add_headers(
                    Authorization = paste("Bearer", access_token),
                    "Content-Type" = "application/json"
                ),
                body = query_json
            )
            
            if (httr::status_code(response) == 200) {
                content <- httr::content(response, "parsed")
                
                if (length(content$data$videos) == 0) {
                    if (length(all_results) == 0) {
                        log_warn(paste("No videos found for description:", desc))
                        return(data.frame())
                    } else {
                        break
                    }
                }
                
                result <- do.call(rbind, lapply(content$data$videos, function(video) {
                    data.frame(
                        video_id = ifelse(is.null(video$id), NA, video$id),
                        create_time = ifelse(is.null(video$create_time), NA, as.POSIXct(video$create_time, origin = "1970-01-01")),
                        region_code = ifelse(is.null(video$region_code), NA, video$region_code),
                        share_count = ifelse(is.null(video$share_count), NA, video$share_count),
                        view_count = ifelse(is.null(video$view_count), NA, video$view_count),
                        like_count = ifelse(is.null(video$like_count), NA, video$like_count),
                        comment_count = ifelse(is.null(video$comment_count), NA, video$comment_count),
                        video_description = ifelse(is.null(video$video_description), NA, video$video_description),
                        username = ifelse(is.null(video$username), NA, video$username),
                        stringsAsFactors = FALSE
                    )
                }))
                
                all_results[[length(all_results) + 1]] <- result
                
                log_info(paste("Retrieved", nrow(result), "videos for description:", desc))
                
                if (!is.null(content$data$has_more) && content$data$has_more) {
                    query_body$cursor <- content$data$cursor
                    query_body$search_id <- content$data$search_id
                } else {
                    log_info("No more results. Returning all retrieved data.")
                    break
                }
            } else if (httr::status_code(response) %in% c(400, 500)) {
                log_warn(paste("Pagination failed with HTTP status", httr::status_code(response), "for description:", desc))
                if (length(all_results) > 0) {
                    log_info("Returning results from successful queries.")
                    break
                } else {
                    return(data.frame())
                }
            } else {
                log_error(paste("API request failed with status code:", httr::status_code(response), "for description:", desc))
                stop(paste("API request failed with status code:", httr::status_code(response)))
            }
        }, error = function(e) {
            if (grepl("daily_quota_limit_exceeded", e$message, ignore.case = TRUE)) {
                log_error("Daily API quota limit exceeded")
                stop("Process terminated: Daily API quota limit exceeded.", call. = FALSE)
            } else {
                log_warn(paste("An unexpected error occurred:", e$message))
                if (attempts >= max_retries) {
                    if (length(all_results) > 0) {
                        log_info("Max retries exceeded. Returning results from successful queries.")
                        break
                    } else {
                        return(data.frame())
                    }
                }
            }
        })
        
        if (attempts >= max_retries) {
            log_warn(paste("Max retries exceeded for description:", desc))
            break
        }
    }
    
    if (length(all_results) > 0) {
        return(do.call(rbind, all_results))
    } else {
        return(data.frame())
    }
}

# Main script to fetch all videos
fetch_all_videos <- function(unique_videos_desc, startdate, enddate, access_token) {
    all_videos_list <- list()
    errors_list <- list()

    log_info("Starting to fetch all videos")

    for (i in 1:nrow(unique_videos_desc)) {
        desc <- unique_videos_desc$video_description[i]
        
        result <- tryCatch({
            temp_videos <- get_videos_by_description(desc, startdate, enddate, access_token, max_retries = 3)
            log_info(paste("Successfully fetched videos for description", i))
            temp_videos
        }, error = function(e) {
            log_error(paste("Error fetching videos for description", i, ":", e$message))
            errors_list[[length(errors_list) + 1]] <- list(description = desc, error = e$message)
            global_error_list <<- c(global_error_list, list(list(type = "video_description", index = i, message = e$message)))
            data.frame()
        })
        
        all_videos_list[[i]] <- result
        Sys.sleep(2)
    }

    all_videos <- bind_rows(all_videos_list)

    # Process the combined data
    all_videos <- all_videos %>%
        {if ("region_code" %in% names(.)) 
            mutate(., region_code = toupper(region_code)) 
         else .} %>%
        filter(video_description != "" & nchar(video_description) >= 80) %>%
        distinct() %>%
        mutate(video_url = ifelse(!is.na(username) & !is.na(video_id),
                                  paste0("https://www.tiktok.com/@", username, "/video/", video_id),
                                  NA))

    log_info(paste("Fetched a total of", nrow(all_videos), "videos"))
    log_info(paste("Encountered", length(errors_list), "errors during fetching"))

    return(list(videos = all_videos, errors = errors_list))
}

# Fetch all videos
all_videos_result <- tryCatch({
    fetch_all_videos(unique_videos_desc, startdate, enddate, access_token)
}, error = function(e) {
    log_error(paste("Error in fetch_all_videos:", e$message))
    global_error_list <<- c(global_error_list, list(list(type = "fetch_all_videos", message = e$message)))
    list(videos = data.frame(), errors = list())
})

# Save the results
write.csv(all_videos_result$videos, file = "latest_snapshot.csv", row.names = FALSE)
```

```{r coordinated_detection, message=FALSE}

tryCatch({
  if(nrow(all_videos_result$videos) > 0 && "video_description" %in% colnames(all_videos_result$videos)) {
    prep_data <- CooRTweet::prep_data(x = all_videos_result$videos,
                                      object_id = "video_description",
                                      account_id = "username",
                                      content_id = "video_id",
                                      timestamp_share = "create_time")

    result <- CooRTweet::detect_groups(x = prep_data,
                                       time_window = 180, # set the time interval
                                       min_participation = 2, # set the minimum number of repetition
                                       remove_loops = T)

    coord_graph <- CooRTweet::generate_coordinated_network(x = result, 
                                                edge_weight = 0.5, # default 0.5
                                                objects = TRUE,
                                                subgraph = 1)

    # Before calculating summary statistics, ensure the 'coord_graph' and 'result' are valid
    if(!is.null(coord_graph) && !is.null(result)) {
      # Check if 'coord_graph' has the expected structure
      if(!("igraph" %in% class(coord_graph))) {
        log_error("coord_graph is not of type igraph")
        stop("coord_graph is not of type igraph. Please check the generate_coordinated_network output.")
      }
      
      # Proceed with summary calculations
      summary_groups <- CooRTweet::group_stats(coord_graph = coord_graph, weight_threshold = "full")
      summary_accounts <- CooRTweet::account_stats(coord_graph = coord_graph, result = result, weight_threshold = "full")
      log_info("Successfully calculated summary statistics")
      
      new_account_ids <- summary_accounts[!(summary_accounts$account_id %in% coord_users), ]
      updated_list <- c(coord_users, new_account_ids$account_id)
    } else {
      log_warn("coord_graph or result is NULL. Skipping summary statistics.")
      new_account_ids <- data.frame()
      updated_list <- account_ids$x
    }
  } else {
    log_warn("No videos available for coordinated detection analysis")
    new_account_ids <- data.frame()
    updated_list <- coord_users
  }
}, error = function(e) {
  log_error(paste("Error in coordinated detection:", e$message))
  global_error_list <<- c(global_error_list, list(list(type = "coordinated_detection", message = e$message)))
  new_account_ids <- data.frame()
  updated_list <- coord_users
})

```

```{r save_output, message=FALSE}

# Define the log file path
log_file_path <- "/home/mine/VERAAI_WP4/output/new_accounts_log.csv"

# Check if the log file exists; if not, create it with a header
if (!file.exists(log_file_path)) {
  write.csv(data.frame(Timestamp = character(), New_Accounts_Count = numeric()), 
            log_file_path, row.names = FALSE)
}

# Prepare the new entry with the current timestamp and the count of new accounts
new_entry <- data.frame(Timestamp = Sys.Date(), New_Accounts_Count = nrow(new_account_ids))

# Append the new entry to the log file using write.table
write.table(new_entry, file = log_file_path, sep = ",", col.names = FALSE, 
            row.names = FALSE, append = TRUE, quote = FALSE)

# Make sure the 'write.csv' operation for updating the account list is also wrapped in tryCatch
tryCatch({
  # Assuming 'updated_list' is properly generated from 'summary_accounts'
  if(exists("updated_list") && length(updated_list) > 0) {
    write.csv(x = updated_list, file = "/home/mine/VERAAI_WP4/lists/tiktok_coordinated_accounts_ids.csv", row.names = FALSE)
    log_info("Successfully updated the list of TikTok coordinated accounts")
  } else {
    log_warn("updated_list is missing or empty. No new data to write.")
    stop("updated_list is missing or empty. No new data to write.")
  }
}, error = function(e) {
  log_error(paste("Failed to write the updated list of TikTok coordinated accounts:", e$message))
  stop("Failed to write the updated list of TikTok coordinated accounts to CSV: ", e$message)
})

```

# Today Output

```{r results_summary, results='asis'}

# Construct the message with available data
message <- sprintf("We attempted to retrieve videos from %s monitored accounts, during the period from %s to %s. ",
                   scales::comma(length(coord_users)), 
                   format(as.Date(startdate), "%B %d, %Y"), 
                   format(as.Date(enddate), "%B %d, %Y"))

if (nrow(recent_videos) > 0) {
    message <- paste0(message, sprintf("We successfully retrieved %s recent videos. ", scales::comma(nrow(recent_videos))))
} else {
    message <- paste0(message, "We were unable to retrieve any recent videos due to API issues. ")
}

message <- paste0(message, sprintf("Using available data, we accessed a total of %s videos posted on TikTok within the timeframe. ",
                                   scales::comma(nrow(all_videos_result$videos))))

if (exists("summary_accounts") && !is.null(summary_accounts)) {
    message <- paste0(message, sprintf("Our analysis for coordinated detection in these videos identified %s accounts spread across %s components, and it also uncovered %s new accounts exhibiting coordinated behavior.",
                                       scales::comma(nrow(summary_accounts)),
                                       scales::comma(igraph::components(coord_graph)$no),
                                       scales::comma(nrow(new_account_ids))))
} else {
    message <- paste0(message, "We were unable to perform coordinated detection analysis due to insufficient data.")
}

cat(message)

# Error summary
cat("\n\nError Summary:\n")
if (length(global_error_list) > 0) {
  cat(paste("Total errors encountered:", length(global_error_list), "\n"))
  error_types <- table(sapply(global_error_list, function(x) x$type))
  cat("Errors by type:\n")
  print(error_types)

  # Display a few sample errors
  cat("\nSample Errors:\n")
  sample_size <- min(5, length(global_error_list))
  sample_errors <- sample(global_error_list, sample_size)
  for (error in sample_errors) {
    cat(paste0("Type: ", error$type, ", Message: ", error$message, "\n"))
  }
} else {
  cat("No errors were encountered during script execution.\n")
}

```

```{r new-accounts-plot, fig.cap="Trend of newly discovered TikTok accounts.", message=FALSE}

log_data <- read_csv(log_file_path)

# Ensure the Timestamp is treated as a Date
log_data$Timestamp <- as.Date(log_data$Timestamp)

# Generate the bar chart
ggplot(log_data, aes(x = Timestamp, y = New_Accounts_Count, group=1)) +
  geom_bar(stat = "identity", fill = "blue") +
  labs(title = "Newly Discovered TikTok Accounts Over Time",
       x = "Date",
       y = "Number of New Accounts") +
  theme_minimal() + # Use a minimal theme
  scale_y_continuous(limits = c(0, NA)) # Ensure y-axis starts at 0

```

Explore the interactive chart. Node click to visit the TikTok account.

```{r viz, message=FALSE}

# Add URL attribute to nodes
V(coord_graph)$url <- paste0("https://tiktok.com/@", V(coord_graph)$name)

# Convert igraph object to visNetwork
data <- toVisNetworkData(coord_graph)

# Plot the graph using visNetwork
network <- visNetwork(data$nodes, data$edges, width = "100%", height = "600px") %>%
  visLayout(randomSeed = 123) # Setting a seed for reproducible layout

# Add event to open URL on node click and log URL for debugging
network <- network %>%
  visEvents(click = "function(params) {
                                      if (params.nodes.length > 0) {
                                        var nodeId = params.nodes[0];
                                        var nodeData = this.body.data.nodes.get(nodeId);
                                        var url = nodeData.url;
                                        console.log('Opening URL:', url);  // For debugging
                                        window.open(url, '_blank');
                                      }
                                    }")

```

```{r interactive-plot, fig.cap="Interactive visualization of coordinated account detected today. Click on a node to visit the respective TikTok account."}
# Print the network
network
```

# About

|                                                                                                                                                                      |                                                                                                                                                                                                                                                                                                           |
|--------------------|----------------------------------------------------|
| [![vera ai logo](https://www.disinfo.eu/wp-content/uploads/elementor/thumbs/vera-logo_black-pz7er90kthmarde380cigj2nwx09ubmujp4y24avw2.jpg)](https://www.veraai.eu/) | [vera.ai](https://www.veraai.eu/home) is a research and development project focusing on disinformation analysis and AI supported verification tools and services. Project funded by EU Horizon Europe, the UK's innovation agency, and the Swiss State Secretariat for Education, Research and Innovation |

# References

Giglietto, F., Marino, G., Mincigrucci, R., & Stanziano, A. (2023). A Workflow to Detect, Monitor, and Update Lists of Coordinated Social Media Accounts Across Time: The Case of the 2022 Italian Election. Social Media + Society, 9(3). https://doi.org/10.1177/20563051231196866