Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add container build/deploy job and improve run pattern #31

Merged
merged 52 commits into from
Jul 19, 2023
Merged

add container build/deploy job and improve run pattern #31

merged 52 commits into from
Jul 19, 2023

Conversation

johnseekins
Copy link
Contributor

@johnseekins johnseekins commented Jun 20, 2023 •
edited
Loading

Start planning towards auto-updating geo data. First step is actually having a container we can run.

Also re-architect the actual update tooling to make it more streamlined, robust, and easier to manage in a container.

John Seekins added 23 commits June 20, 2023 15:43
add container build/deploy job
5f32496 
Signed-off-by: John Seekins <[email protected]>
upstream (openstates-core) is still 3.9, so we'll stick with that
4c1207b 
Signed-off-by: John Seekins <[email protected]>
correct typo, update tippecanoe version and remove some packages befo…
d0dbb77 
…re deploy

Signed-off-by: John Seekins <[email protected]>
better build of geo
2ea394b 
Signed-off-by: John Seekins <[email protected]>
Remove post-redistricting warning
15d7126 
Signed-off-by: John Seekins <[email protected]>
remove building DATABASE_URL env
100c63b 
Signed-off-by: John Seekins <[email protected]>
increase zoom back to what it was
ae87c0a 
Signed-off-by: John Seekins <[email protected]>
correct ids for territories
cada6aa 
Signed-off-by: John Seekins <[email protected]>
add version file for pyenv
c8e90f7 
Signed-off-by: John Seekins <[email protected]>
update dependencies
8c888d5 
Signed-off-by: John Seekins <[email protected]>
one script to execute all steps
b044c91 
Signed-off-by: John Seekins <[email protected]>
add missing files
e4b22f8 
Signed-off-by: John Seekins <[email protected]>
more functions into utils
f0bd512 
Signed-off-by: John Seekins <[email protected]>
fix some import issues
0fd48b3 
Signed-off-by: John Seekins <[email protected]>
set environment variable correctly
4b337fa 
Signed-off-by: John Seekins <[email protected]>
update openstates dep
a374f95 
Signed-off-by: John Seekins <[email protected]>
possibly working Mapbox API calls?
e9f7cbb 
Signed-off-by: John Seekins <[email protected]>
fix linting
123f4d7 
Signed-off-by: John Seekins <[email protected]>
actually call new function
2cf276d 
Signed-off-by: John Seekins <[email protected]>
actually check for missing env vars and make bulk upload access more …
5402c75 
…flexible

Signed-off-by: John Seekins <[email protected]>
update docs to reflect new environment variables
6978120 
Signed-off-by: John Seekins <[email protected]>
working migrations
48cee0e 
Signed-off-by: John Seekins <[email protected]>
don't try to download files that don't exist
5eddae8 
Signed-off-by: John Seekins <[email protected]>
@johnseekins johnseekins changed the title add container build/deploy job add container build/deploy job and improve run pattern Jul 13, 2023
John Seekins added 4 commits July 13, 2023 09:30
add missing files
0e7caaa 
Signed-off-by: John Seekins <[email protected]>
remove un-needed files and don't check for shell when we have no shel…
2f42c6c 
…l scripts

Signed-off-by: John Seekins <[email protected]>
remove git after build
b7a1cc9 
Signed-off-by: John Seekins <[email protected]>
add comment explaining removal
2b958f7 
Signed-off-by: John Seekins <[email protected]>
@johnseekins johnseekins marked this pull request as ready for review July 13, 2023 15:38
correct docker command
6b56d1c 
Signed-off-by: John Seekins <[email protected]>
John Seekins added 2 commits July 13, 2023 09:43
follow Mapbox API examples a little more closely
77ee10b 
Signed-off-by: John Seekins <[email protected]>
fix linting
eb50612 
Signed-off-by: John Seekins <[email protected]>
John Seekins added 14 commits July 14, 2023 07:55
fix docker build
adc57da 
Signed-off-by: John Seekins <[email protected]>
update openstates
576b73c 
Signed-off-by: John Seekins <[email protected]>
make sure we don't have transient AWS tokens messing up steps
8074ec3 
Signed-off-by: John Seekins <[email protected]>
updates to documentation
e632b10 
Signed-off-by: John Seekins <[email protected]>
handle env keys not being there
8176420 
Signed-off-by: John Seekins <[email protected]>
small updates that don't matter
d8b85ad 
Signed-off-by: John Seekins <[email protected]>
don't re-generate tiles if they exist
417b62a 
Signed-off-by: John Seekins <[email protected]>
remove db settings
daba6bd 
Signed-off-by: John Seekins <[email protected]>
start working out how to remove old divisions
c5aebb3 
Signed-off-by: John Seekins <[email protected]>
cleanup command works correctly, use absolute paths
c011e4e 
Signed-off-by: John Seekins <[email protected]>
fix linting
de33fda 
Signed-off-by: John Seekins <[email protected]>
remove duplicate path formatting
819f42f 
Signed-off-by: John Seekins <[email protected]>
update exceptions and actually delete
225d15f 
Signed-off-by: John Seekins <[email protected]>
better handling of password for psql test and slightly better logging
c543e36 
Signed-off-by: John Seekins <[email protected]>
@johnseekins
Copy link
Contributor Author

Division deletion should work as of c543e36:

Checking for any divisions we should remove...
Loaded 7293 local divisions for comparison
Found 1 divisions not stored locally
Deleting 1 divisions from DB

John Seekins added 8 commits July 18, 2023 15:59
actually add bounds as required by docs
bb8b35c 
Signed-off-by: John Seekins <[email protected]>
fix linting and slightly more typing
c920255 
Signed-off-by: John Seekins <[email protected]>
correct names of tilesets
3f9dc18 
Signed-off-by: John Seekins <[email protected]>
fix small bugs for full territory support (TIGER doesn't provide sldu…
89c8788 
…/sldl boundaries for most territories)

Signed-off-by: John Seekins <[email protected]>
remove un-supported chambers from AS
dbc6f7a 
Signed-off-by: John Seekins <[email protected]>
more logging during mapbox upload
50a7631 
Signed-off-by: John Seekins <[email protected]>
update readme to give some tips on using environment files
1fd594b 
Signed-off-by: John Seekins <[email protected]>
actually delete old divisions
098b03f 
Signed-off-by: John Seekins <[email protected]>
@johnseekins johnseekins merged commit cea77f2 into openstates:main Jul 19, 2023
1 check passed
@johnseekins johnseekins deleted the containerize branch July 19, 2023 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NewAgeAirbender NewAgeAirbender NewAgeAirbender approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@johnseekins @NewAgeAirbender