Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

General handler for open source data preprocessing #1302

Merged
merged 7 commits into from
Oct 12, 2022

Conversation

Arthur-Null
Copy link
Contributor

Description

Motivation and Context

How Has This Been Tested?

  • Pass the test by running: pytest qlib/tests/test_all_pipeline.py under upper directory of qlib.
  • If you are adding a new feature, test on your own test scripts.

Screenshots of Test Results (if appropriate):

  1. Pipeline test:
  2. Your own tests:

Types of changes

  • Fix bugs
  • Add new feature
  • Update documentation

qlib/contrib/data/highfreq_handler.py Outdated Show resolved Hide resolved
qlib/contrib/data/highfreq_handler.py Outdated Show resolved Hide resolved
@@ -124,14 +124,37 @@ def __init__(
day_length=240,
):
self.day_length = day_length

def check_transform_proc(proc_l):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function duplicates with the previous one in this file. Pleases shared them

@@ -124,14 +124,37 @@ def __init__(
day_length=240,
):
self.day_length = day_length

def check_transform_proc(proc_l):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't this function be used here

def check_transform_proc(proc_l, fit_start_time, fit_end_time):

@you-n-g you-n-g merged commit 54928e9 into microsoft:main Oct 12, 2022
@you-n-g you-n-g added the enhancement New feature or request label Dec 9, 2022
qianyun210603 pushed a commit to qianyun210603/qlib that referenced this pull request Mar 23, 2023
* feat(data): ✨ add a general highfreq data handler for open source

Add HighFreqOpenHandler and HighFreqOpenBacktestHandler for data pipeline without paused_num
information.

* fix: position of parameter init

* style(data): 💄 rename open to general

* style(data): 💄 lint

* style: 💄 delete useless comment & fix inheritance relation

* style: 💄 lint

* style: 💄 remove duplicated function

Co-authored-by: mingzhehan <v-zhaoxing@Microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants