Transform XBRL core_ferc714__yearly_planning_area_demand_forecast table

[aesharpe]

Overview

Closes #3837 .

What problem does this address?

Adds XBRL data (2021 +) for the annual demand forecast table.

What did you change?

• Add a YearlyPlanningAreaDemandForecast class and class functions.
• Migrate CSV transforms into these class functions.
• Add forecast to the column names with forecasted values.

Ideas

We could add some tests for the 714 data that made sure the mwh and mw values fell within a reasonable range. Might be something we put on the backburner for now.

Testing

How did you make sure this worked? How can a reviewer verify this?

Materialize the core_ferc714__yearly_planning_area_demand_forecast table in dagster.

To-do list

Beta Give feedback

1. If updating analyses or data processing functions: make sure to update or write data validation tests (e.g. test_minmax_rows())
2. Update the release notes: reference the PR and related issues.
3. Run make pytest-coverage locally to ensure that the merge queue will accept your PR.
4. Review the PR yourself and call out any questions or issues you have
5. For minor ETL changes or data additions, once make pytest-coverage passes, make sure you have a fresh full PUDL DB downloaded locally, materialize new/changed assets and all their downstream assets and run relevant data validation tests using pytest and --live-dbs.
6. For bigger ETL or data changes run the full ETL locally and then run the data validations using make pytest-validate.
7. Alternatively, run the build-deploy-pudl GitHub Action manually.

[aesharpe]

Just a high-level comment, but I spent some time tackling the duplicate PK values in the XBRL data before I realized that they were getting handled in the FercXBRLSQLiteIOManager class load_input function. Specifically, I didn't realize that the older publication_date values were getting dropped and the report_year column was getting made there. It's a little confusing to have that happen in the IOManager vs. the transforms. I understand that it might be more efficient to do it that way though.

Out of scope to change for this PR, just a thought I had.

[aesharpe]

The hourly table has an out prefix. Which one should we have here?