YNR can't sync deleted ballot-level elections on EE

[awdem]

If a ballot-level election is deleted on EE, YNR's EE importer can't currently find out about it. This is causing the sites to be out of sync.

The issue is caused by the fact that the command that syncs YNR with EE (uk_create_elections_from_every_election) looks for top-level elections then looks for their children.

In the command:

delete_deleted_elections only syncs deleted top-level elections and then looks for their deleted child ballots:

yournextrepresentative/ynr/apps/elections/uk/management/commands/uk_create_elections_from_every_election.py

Lines 120 to 139 in 0096b0f

	def delete_deleted_elections(self, recently_updated_timestamp):
	# Get all deleted elections from EE
	params = {
	"poll_open_date__gte": str(date.today() - timedelta(days=30)),
	"deleted": 1,
	}
	if recently_updated_timestamp:
	params.pop("poll_open_date__gte")
	params["modified"] = recently_updated_timestamp
	ee_importer = EveryElectionImporter(params)
	# TODO account for -recently-updated flag herw?
	ee_importer.build_election_tree(deleted=True)
	for ballot_id, election_dict in ee_importer.ballot_ids.items():
	election_dict.delete_ballot()
	for group_id, election_dict in ee_importer.group_ids.items():
	election_dict.delete_election()

import_approved_elections only syncs top-level elections and then looks for their non-deleted child ballots.

yournextrepresentative/ynr/apps/elections/uk/management/commands/uk_create_elections_from_every_election.py

Lines 59 to 89 in 0096b0f

	def import_approved_elections(
	self, full=False, poll_open_date=None, recently_updated_timestamp=None
	):
	# Get all approved elections from EveryElection
	query_args = None
	if full:
	query_args = {}
	if poll_open_date:
	query_args = {"poll_open_date": poll_open_date}
	if recently_updated_timestamp:
	query_args = {"modified": recently_updated_timestamp}
	ee_importer = EveryElectionImporter(query_args)
	ee_importer.build_election_tree()
	for ballot_id, election_dict in ee_importer.ballot_ids.items():
	try:
	parent = ee_importer.get_parent(ballot_id)
	except KeyError as e:
	# raise the exception if this is not a recent update
	if not recently_updated_timestamp:
	raise e
	# check the parent election already exists in the DB and if it
	# doesn't allow an exception to be raised
	Election.objects.get(slug=election_dict.parent)
	# otherwise set parent to None as the KeyError indicates it is
	# not in the election tree because there is nothing to update
	parent = None
	election_dict.get_or_create_ballot(parent=parent)

In the case of a top-level election with a deleted ballot:

• delete_deleted_elections won't find the deleted ballot because its parent is not deleted
• import_approved_elections won't find the deleted ballot, even if it finds the top level election, because it only looks for non-deleted ballots

They both pass parameters to EveryElectionWrapper's build_election_tree to query the EE api:

yournextrepresentative/ynr/apps/elections/uk/every_election.py

Line 369 in 0096b0f

def build_election_tree(self, deleted=False):

No matter what other parameters are passed, build_election_tree always queries for top-level elections first, then queries for the ballots of those elections:

yournextrepresentative/ynr/apps/elections/uk/every_election.py

Lines 383 to 385 in 0096b0f

	# First pass: just get elections
	self.query_args["identifier_type"] = "election"
	url = self.build_url_with_query_args()

There are probably a couple ways to address this but the simplest might be changing delete_deleted_elections so that it also looks for deleted ballots.

Related issue on EE: DemocracyClub/EveryElection#2289

[awdem]

Update:

because YNR only syncs parent elections recently modified on EE, it doesn't pick up on changes made on EE to ballot-level elections including creation and approval. This caused another case of desync when two ballot-level elections were made on EE a week or so after their parent election was created. They didn't show up in YNR until I went and saved the parent election so that is modified timestamp would change.