Merge pull request #337 from fedspendingtransparency/stg

stg --> master

loading_data.md

@@ -50,3 +50,11 @@ Run the following Django management command to load a single submission from the
 **Note:** If the specified submission has already been loaded, this command will delete the existing records first. Thus, the `load_submission` command is safe to re-run.
 
     `python manage.py load_submission [broker submission_id]`
+
+### Other data loading related commands
+
+* `python manage.py update_location_usage_flags` - Updates all locations to have proper usage flags. This should be run after any set of submission loads to ensure the flags are properly set.
+
+* `python manage.py load_executive_compensation --all` - Loads executive compensation data for any currently loaded submissions. For more information on other options for this command, reference the command's help text.
+
+* `python manage.py load_subawards --all` - Loads subwards for any currently loaded submissions. For more information on other options for this command, reference the command's help text. Note that this command only needs to be run when subaward data has been updated after the original submission has been loaded; during the normal submission loading process the current set of subawards will be loaded.

usaspending_api/api_docs/markdown/api_tutorial.md

@@ -25,13 +25,13 @@
 
 # Introductory Tutorial <a name="introduction"></a>
 
-Welcome to the introductory USASpending API tutorial. This tutorial is designed for people who aren't familiar with APIs and how to use them. If you already know what an "endpoint" is and the difference between GET and POST, you'll want to visit [Using the API](/docs/using-the-api), or jump right into the data with some [request recipes](/docs/recipes/).
+Welcome to the introductory USASpending API tutorial. This tutorial is designed for people who aren't familiar with APIs and how to use them. If you already know what an _endpoint_ is and the difference between `GET` and `POST`, you'll want to visit [Using the API](/docs/using-the-api) or jump right into the data with some [request recipes](/docs/recipes/).
 
 ## What's an API? <a name="whats-an-api"></a>
 
-"API" stands for "Application Programmer Interface." APIs make it easy for computer programs to request and recieve information in a format they can understand.
+_API_ stands for _Application Programmer Interface_. APIs make it easy for computer programs to request and receive information in a format they can understand.
 
-If you're looking for federal spending data that's designed to be read by humans instead of computer programs, you should head to <a href="https://www.usaspending.gov/Pages/Default.aspx">usaspending.gov</a>, or visit <a href="https://openbeta.usaspending.gov">openbeta.usaspending.gov</a> for information on the updated version of the site that's currently in development.
+If you're looking for federal spending data that's designed to be read by humans instead of computers, you should head to <a href="https://beta.usaspending.gov">our website</a>.
 
 ## Using the API <a name="using-the-api"></a>
 
@@ -41,7 +41,7 @@ You do not need to complete this tutorial in its entirety to get started. Feel f
 
 ## Endpoint Overview <a name="endpoint-overview"></a>
 
-When you type a url into your browser, it usually returns a web page: a document that your browser knows how to display for you to read. APIs use urls, too--but instead of returning formated web pages, API urls return data structured to be easy for computers to parse. API urls are called "endpoints." Just as many webpages make up a web site, many endpoints make up an API.
+When you type a url into your browser, it usually returns a web page: a document that your browser knows how to display for you to read. APIs use urls, too--but instead of returning formatted web pages, API urls return data structured to be easy for computers to parse. API urls are called _endpoints_. Just as many webpages make up a web site, many endpoints make up an API.
 
 The USASpending API supports a number of endpoints. For example `/api/v1/awards/` is our awards endpoint.
 
@@ -61,7 +61,7 @@ For a comprehensive list of endpoints and their data, please see the [USASpendin
 
 #### Responses
 
-Responses for data endpoints are json objects and follow the same structure, generally speaking:
+Responses for data endpoints are JSON objects and follow the same structure, generally speaking:
 
 ```
 {
@@ -101,7 +101,7 @@ This GET request would return awards where the <span title="A code identifying t
 
 ##### POST Filtering
 
-Complex filters can be constructed using POST requests. Let's construct a json object we can POST to search for awards with signing dates between June 1st 2016 and June 1st 2017. First, let's take a look at an empty post request.
+Complex filters can be constructed using POST requests. Let's construct a JSON object we can POST to search for awards with signing dates between June 1st 2016 and June 1st 2017. First, let's take a look at an empty post request.
 
 ```
 {
@@ -345,7 +345,7 @@ Let's try it with `/api/v1/awards/total`.
 }
 ```
 
-Here we specify the `field` as `total_obligation` and that we're grouping by each record's signing date's fiscal year. The response we recieve looks something like this:
+Here we specify the `field` as `total_obligation` and that we're grouping by each record's signing date's fiscal year. The response we receive looks something like this:
 
 ```
 {

usaspending_api/api_docs/markdown/data_dictionary.md

@@ -16,7 +16,6 @@
 </ul>
 
 [//]: # (Begin Content)
-**Please note, the data dictionary is under heavy development. As a result, some information may be missing**
 ## Data Dictionary <a name="datadictionary"></a>
 
 Federal spending has a vocabulary all its own. This page is under construction, but will strive to help you understand the most commonly requested endpoints and fields returned.

usaspending_api/api_docs/markdown/endpoints.md

@@ -18,9 +18,7 @@
 
 # API Endpoints
 
-This page is intended for users who are already familiar with APIs. If you're not sure what "endpoint" means, and what GET and POST requests are, you may find the [introductory tutorial](/docs/intro-tutorial) more useful to start.
-
-While the API is under development, we are gradually increasing the amount of available data, which is currently limited to a few federal agency submissions and small slices of USAspending.gov historical data.
+This page is intended for users who are already familiar with APIs. If you're not sure what _endpoint_ means, and what `GET` and `POST` requests are, you may find the [introductory tutorial](/docs/intro-tutorial) more useful to start.
 
 Endpoints do not currently require any authorization.
 
@@ -39,34 +37,41 @@ To reduce unnecessary data transfer, most endpoints return a default set of info
 
 ## Endpoint Index <a name="endpoint-index"></a>
 
-| Endpoint | Methods | Response Object | Data
-| -------- | ---: | ------ | ------ |
-| [/api/v1/awards/](/api/v1/awards/) | GET, POST | <a href="#award">Awards</a> | Returns a list of award records |
-| /api/v1/awards/:id | GET, POST | <a href="#award">Award</a> | Returns a single award records with all fields |
-| [/api/v1/awards/autocomplete/](/api/v1/awards/autocomplete/) | POST | Autocomplete (see [Using the API](/docs/using-the-api))| Supports autocomplete on award records |
-| [/api/v1/awards/total/](/api/v1/awards/total/) | POST |  Aggregate (see [Using the API](/docs/using-the-api)) | Supports aggregation on award records |
-| [/api/v1/federal_accounts/](/api/v1/federal_accounts/) | GET, POST | <a href="#federal-account">Federal Account</a> | Returns a list of federal accounts |
-| [/api/v1/federal_accounts/autocomplete/](/api/v1/federal_accounts/autocomplete/) | POST | Autocomplete (see [Using the API](/docs/using-the-api))| Supports autocomplete on federal account records |
-| [/api/v1/tas/balances/](/api/v1/tas/balances/) | GET, POST | <a href="#appropriation-account">Yearly Appropriation Account Balances</a> | Returns a list of appropriation account balances by fiscal year |
-| [/api/v1/tas/balances/total/](/api/v1/tas/balances/total/) | POST |  Aggregate (see [Using the API](/docs/using-the-api)) | Supports aggregation on appropriation account records |
-| [/api/v1/tas/balances/quarters/](/api/v1/tas/balances/quarters/) | GET, POST | <a href="#appropriation-account-balances-quarterly">Quarterly Appropriation Account Balances</a> | Returns a list of appropriation account balances by fiscal quarter|
-| [/api/v1/tas/balances/quarters/total/](/api/v1/tas/balances/quarters/total/) | POST |  Aggregate (see [Using the API](/docs/using-the-api)) | Supports aggregation on quarterly appropriation account records |
-| [/api/v1/tas/categories/](/api/v1/tas/categories/) | GET, POST | <a href="#accounts-prg-obj">Yearly Appropriation Account Balances (by Category)</a> | Returns a list of appropriation account balances by fiscal year broken up by program activities and object class |
-| [/api/v1/tas/categories/total/](/api/v1/tas/categories/total/) | POST |  Aggregate (see [Using the API](/docs/using-the-api)) | Supports aggregation on appropriation account (by category) records |
-| [/api/v1/tas/categories/quarters/](/api/v1/tas/categories/quarters/) | GET, POST | <a href="#accounts-prg-obj-quarterly">Quarterly Appropriation Account Balances (by Category)</a> | Returns a list of appropriation account balances by fiscal quarter broken up by program activities and object class |
-| [/api/v1/tas/categories/quarters/total/](/api/v1/tas/categories/quarters/total/) | POST |  Aggregate (see [Using the API](/docs/using-the-api)) | Supports aggregation on quarterly appropriation account (by category) records |
-| [/api/v1/tas/](/api/v1/tas/) | GET, POST | <a href="#tas">Treasury Appropriation Account</a> | Returns a list of treasury appropriation accounts, by TAS |
-| [/api/v1/tas/autocomplete/](/api/v1/tas/autocomplete/) | POST | Autocomplete (see [Using the API](/docs/using-the-api))| Supports autocomplete on TAS records |
-| [/api/v1/accounts/awards/](/api/v1/accounts/awards/) | GET, POST | <a href="#accounts-by-award">Financial Accounts (by Award)</a> | Returns a list of financial account data grouped by TAS and broken up by Program Activity and Object Class codes |
-| /api/v1/accounts/awards/:id | GET, POST | <a href="#accounts-by-award">Financial Account (by Award)</a> | Returns a single financial account record, grouped by TAS, with all fields |
-| [/api/v1/transactions/](/api/v1/transactions/) | GET, POST | <a href="#transaction">Transaction</a> | Returns a list of transactions - contracts, grants, loans, etc. |
-| /api/v1/transactions/:id | GET, POST | <a href="#transaction">Transaction</a> | Returns a single transaction record with all fields |
-| [/api/v1/transactions/total/](/api/v1/transactions/total/) | POST | Aggregate (see [Using the API](/docs/using-the-api)) | Supports aggregation on transaction records |
-| [/api/v1/references/locations/](/api/v1/references/locations/) | POST | <a href="#locations">Location</a> | Returns a list of locations - places of performance or vendor locations |
-| [/api/v1/references/locations/geocomplete/](/api/v1/references/locations/geocomplete/) | POST | Location Hierarchy (see [Using the API](/docs/using-the-api)) | Supports geocomplete queries, see [Using the API](/docs/using-the-api) |
-| [/api/v1/references/agency/](/api/v1/references/agency/) | GET, POST | <a href="#agencies">Agency</a> | Returns a list of agency records |
-| [/api/v1/references/agency/autocomplete/](/api/v1/references/agency/autocomplete/) | POST | Autocomplete (see [Using the API](/docs/using-the-api)) | Supports autocomplete on agency records |
-| [/api/v1/references/cfda/](/api/v1/references/cfda/) | GET, POST | <a href="#cfda-programs">CFDA Programs</a> | Returns a list of CFDA Programs |
-| /api/v1/references/cfda/:id | GET, POST | <a href="#cfda-programs">CFDA Program</a> | Returns a single CFDA program, with all fields |
-| [/api/v1/references/recipients/autocomplete/](/api/v1/references/recipients/autocomplete/) | POST | Autocomplete (see [Using the API](/docs/using-the-api)) | Supports autocomplete on recipient records |
-| [/api/v1/submissions/](/api/v1/submissions/) | GET, POST | <a href="#submissions">SubmissionAttributes</a> | Returns a list of submissions |
+| Endpoint | Methods | Description |
+| -------- | ------- | ----------- |
+| [/api/v1/accounts/awards/](/api/v1/accounts/awards/) | GET, POST | Returns a list of financial account data by treasury account symbol, program activity, object class, and award |
+| /api/v1/accounts/awards/:id | GET, POST | Returns a single treasury account symbol/program activity/object class/award record with all fields |
+| [/api/v1/accounts/awards/total/](/api/v1/accounts/awards/total/) | POST | Supports aggregation on treasury account symbol/program activity/object class/award records |
+| [/api/v1/awards/](/api/v1/awards/) | GET, POST | Returns a list of award records |
+| /api/v1/awards/:id | GET, POST | Returns a single award record with all fields |
+| [/api/v1/awards/autocomplete/](/api/v1/awards/autocomplete/) | POST | Supports autocomplete on award records |
+| [/api/v1/awards/total/](/api/v1/awards/total/) | POST | Supports aggregation on award records |
+| [/api/v1/federal_accounts/](/api/v1/federal_accounts/) | GET, POST | Returns a list of federal accounts |
+| /api/v1/federal_accounts/:id | GET, POST | Returns a single federal account record with all fields |
+| [/api/v1/federal_accounts/autocomplete/](/api/v1/federal_accounts/autocomplete/) | POST | Supports autocomplete on federal account records |
+| [/api/v1/tas/](/api/v1/tas/) | GET, POST | Returns a list of treasury appropriation accounts (TAS) |
+| /api/v1/tas/:id | GET, POST | Returns a single treasury appropriation account record with all fields |
+| [/api/v1/tas/autocomplete/](/api/v1/tas/autocomplete/) | POST | Supports autocomplete on TAS records |
+| [/api/v1/tas/balances/](/api/v1/tas/balances/) | GET, POST | Returns a list of appropriation account balances by fiscal year |
+| [/api/v1/tas/balances/total/](/api/v1/tas/balances/total/) | POST | Supports aggregation on appropriation account records |
+| [/api/v1/tas/balances/quarters/](/api/v1/tas/balances/quarters/) | GET, POST | Returns a list of appropriation account balances by fiscal quarter|
+| [/api/v1/tas/balances/quarters/total/](/api/v1/tas/balances/quarters/total/) | POST | Supports aggregation on quarterly appropriation account records |
+| [/api/v1/tas/categories/](/api/v1/tas/categories/) | GET, POST | Returns a list of appropriation account balances by fiscal year broken up by program activities and object class |
+| [/api/v1/tas/categories/total/](/api/v1/tas/categories/total/) | POST | Supports aggregation on appropriation account (by category) records |
+| [/api/v1/tas/categories/quarters/](/api/v1/tas/categories/quarters/) | GET, POST | Returns a list of appropriation account balances by fiscal quarter broken up by program activities and object class |
+| [/api/v1/tas/categories/quarters/total/](/api/v1/tas/categories/quarters/total/) | POST | Supports aggregation on quarterly appropriation account (by category) records |
+| [/api/v1/subawards/](/api/v1/subawards/) | GET, POST | Returns a list of subaward records |
+| /api/v1/subawards/:id | GET, POST | Returns a single subaward record with all fields |
+| [/api/v1/subawards/autocomplete/](/api/v1/subawards/autocomplete/) | POST | Supports autocomplete on subawards |
+| [/api/v1/subawards/total/](/api/v1/subawards/total/) | POST | Supports aggregation on subawards |
+| [/api/v1/transactions/](/api/v1/transactions/) | GET, POST | Returns a list of transactions - contracts, grants, loans, etc. |
+| /api/v1/transactions/:id | GET, POST | Returns a single transaction record with all fields |
+| [/api/v1/transactions/total/](/api/v1/transactions/total/) | POST | Supports aggregation on transaction records |
+| [/api/v1/references/agency/](/api/v1/references/agency/) | GET, POST | Returns a list of agency records |
+| [/api/v1/references/agency/autocomplete/](/api/v1/references/agency/autocomplete/) | POST | Supports autocomplete on agency records |
+| [/api/v1/references/cfda/](/api/v1/references/cfda/) | GET, POST | Returns a list of CFDA Programs |
+| /api/v1/references/cfda/:id | GET, POST | Returns a single CFDA program, with all fields |
+| [/api/v1/references/locations/](/api/v1/references/locations/) | POST | Returns a list of locations - places of performance or vendor locations |
+| [/api/v1/references/locations/geocomplete/](/api/v1/references/locations/geocomplete/) | POST | Supports geocomplete queries, see [Using the API](/docs/using-the-api) |
+| [/api/v1/references/recipients/autocomplete/](/api/v1/references/recipients/autocomplete/) | POST | Supports autocomplete on recipient records |
+| [/api/v1/submissions/](/api/v1/submissions/) | GET, POST | Returns a list of submissions |

usaspending_api/api_docs/markdown/landing_page.md

@@ -12,17 +12,15 @@
 
 The USAspending API (Application Programming Interface) allows the public to access comprehensive U.S. government spending data.
 
-The data include spending on "awards" (e.g.,who received federal contracts or grants, amounts, geographic breakdowns, agency breakdowns, and much more). The data also include non-award at the account level, like federal employee compensation, You can learn more about the data and the federal law that requires it to be publicly accessible (the DATA Act) at http://fedspendingtransparency.github.io.
-
-_NOTE:  While this API is under development, we are gradually increasing the amount of available data and the capabilities of the API. Data in the current API comes from a selection of agency submissions and the current USASpending.gov._
+The data include spending on _awards_ (_e.g._, who received federal contracts or grants, geographic breakdowns, agency breakdowns, and much more). The data also include account-level, non-award spending such as federal employee compensation. You can learn more about the data and the federal law that requires it to be publicly accessible (the DATA Act) at http://fedspendingtransparency.github.io.
 
 ## Getting Started <a name="getting-started"></a>
 
-**New to APIs?** Check out our [introductory tutorial](/docs/intro-tutorial) on the API to learn the ropes
+**New to APIs?** Check out our [introductory tutorial](/docs/intro-tutorial) on the API to learn the ropes.
 
-**Familiar with APIs?** Familiar with APIs? See the [Using this API](/docs/using-the-api) section for how to request specific sets of data, and examples of filters. OR jump right into the data with our [list of available Endpoints](/docs/endpoints).
+**Familiar with APIs?** Familiar with APIs? See the [Using this API](/docs/using-the-api) section for how to request specific sets of data and examples of filters. OR jump right into the data with our [list of available Endpoints](/docs/endpoints).
 
-**New to government financial data?** Check out our [data dictionary](/docs/data-dictionary) to learn to navigate it
+**New to government financial data?** Check out our [data dictionary](/docs/data-dictionary) to learn to navigate it.
 
 ## Background <a name="background"></a>
 

usaspending_api/api_docs/markdown/request_recipes.md

@@ -97,4 +97,4 @@ POST
 
 # Postman Collections <a name="postman"></a>
 
-[Postman](https://www.getpostman.com/) is a free app for making easy API requests. You can also use it to import and inspect a collection of pre-generated API requests. [Here is a postman collection](https://raw.githubusercontent.com/fedspendingtransparency/usaspending-api/master/usaspending_api/static_doc_files/docs/usaspending_searchpage_postmancollection.json) you can use to see how we generate the visualizations on [the prototype search page](https://spendingdata.us/#/search/).
+[Postman](https://www.getpostman.com/) is a free app for making easy API requests. You can also use it to import and inspect a collection of pre-generated API requests. [Here is a postman collection](https://raw.githubusercontent.com/fedspendingtransparency/usaspending-api/master/usaspending_api/static_doc_files/docs/usaspending_searchpage_postmancollection.json) you can use to see how we generate the visualizations on [the search page](https://beta.usaspending.gov/#/search/).