if the last processed date doesn't exist, shouldn't we pick a much earlier date and just two weeks ago?

[tomlue]

OpenStemDocs/stages/python/01_get_open_access_pdfs_duckdb.py

Line 16 in dde9bfc

def get_last_processed_date():

def get_last_processed_date():
    last_processed_file = Path('last_processed_date.txt')
    if last_processed_file.exists():
        with open(last_processed_file, 'r') as file:
            return datetime.strptime(file.read().strip(), "%Y-%m-%d")
    else:
        print("No last processed date found, using 2 weeks before today.")
        return date.today() - timedelta(days=14)