Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Online beat tracking #238

Merged
merged 9 commits into from
Apr 13, 2017
Merged

Online beat tracking #238

merged 9 commits into from
Apr 13, 2017

Conversation

superbock
Copy link
Collaborator

@superbock superbock commented Dec 6, 2016

Rebased everything on top of #185.

Before this PR can be merged, a couple of things need to be addressed:

  • add uni-directional RNN beat tracking models
  • make a decision whether to keep the current behaviour of the process() method

@superbock superbock force-pushed the online_beat_tracking branch 2 times, most recently from 4e9b7dc to 9acce81 Compare December 7, 2016 13:00
@superbock superbock force-pushed the online_beat_tracking branch 4 times, most recently from 396a526 to 9a12ea8 Compare December 10, 2016 14:23
@superbock superbock changed the base branch from save_rnn_prev_out_and_state to streaming_mode January 23, 2017 12:35
@superbock superbock force-pushed the streaming_mode branch 3 times, most recently from 109d05b to 6a525d1 Compare February 1, 2017 10:18
@superbock superbock force-pushed the online_beat_tracking branch 2 times, most recently from 30c1710 to 576a884 Compare February 3, 2017 03:00
@superbock superbock mentioned this pull request Feb 13, 2017
17 tasks
@superbock superbock changed the base branch from streaming_mode to master February 24, 2017 13:35
@superbock superbock added this to the v0.15 milestone Feb 26, 2017
@superbock superbock changed the title [WIP] Online beat tracking Online beat tracking Mar 2, 2017
@superbock superbock requested a review from fdlm March 2, 2017 10:25
@winchell
Copy link

winchell commented Mar 5, 2017

Looking forward to using this! I'm testing it out but here's what happens:

Mac-Pro:~ $ DBNBeatTracker online test.wav 
/Library/Python/2.7/site-packages/madmom/processors.py:750: UserWarning: make sure that the `frame_size` (2048) is equal to the maximum value used by any `FramedSignalProcessor`.
  frame_size)
Traceback (most recent call last):
  File "/usr/local/bin/DBNBeatTracker", line 112, in <module>
    main()
  File "/usr/local/bin/DBNBeatTracker", line 108, in main
    args.func(processor, **vars(args))
  File "/Library/Python/2.7/site-packages/madmom/processors.py", line 762, in process_online
    _process((processor, frame, outfile, process_args))
  File "/Library/Python/2.7/site-packages/madmom/processors.py", line 200, in _process
    return process_tuple[0](*process_tuple[1:-1], **process_tuple[-1])
  File "/Library/Python/2.7/site-packages/madmom/processors.py", line 128, in __call__
    return self.process(*args, **kwargs)
  File "/Library/Python/2.7/site-packages/madmom/processors.py", line 491, in process
    data = _process((self.in_processor, data, kwargs))
  File "/Library/Python/2.7/site-packages/madmom/processors.py", line 200, in _process
    return process_tuple[0](*process_tuple[1:-1], **process_tuple[-1])
  File "/Library/Python/2.7/site-packages/madmom/processors.py", line 128, in __call__
    return self.process(*args, **kwargs)
  File "/Library/Python/2.7/site-packages/madmom/processors.py", line 336, in process
    data = _process((processor, data, kwargs))
  File "/Library/Python/2.7/site-packages/madmom/processors.py", line 200, in _process
    return process_tuple[0](*process_tuple[1:-1], **process_tuple[-1])
  File "/Library/Python/2.7/site-packages/madmom/processors.py", line 128, in __call__
    return self.process(*args, **kwargs)
  File "/Library/Python/2.7/site-packages/madmom/processors.py", line 336, in process
    data = _process((processor, data, kwargs))
  File "/Library/Python/2.7/site-packages/madmom/processors.py", line 203, in _process
    return process_tuple[0](*process_tuple[1:-1])
  File "/Library/Python/2.7/site-packages/madmom/ml/nn/__init__.py", line 40, in average_predictions
    predictions = predictions[0]
IndexError: list index out of range

@superbock
Copy link
Collaborator Author

Did you update the models? Please run git submodule update.

@winchell
Copy link

winchell commented Mar 5, 2017

I did try that (but it didn't work):

Mac-Pro:madmom $ git submodule update --force --remote
Submodule path 'madmom/models': checked out '9b7b11dde6f586e2b23e9432d42183c9b2fd1758'

@superbock
Copy link
Collaborator Author

Ah, I forgot to include the models in setup.py to be installed as well. Try changing line 40 in setup.py to 'models/beats/201[56]/*'.

@winchell
Copy link

winchell commented Mar 5, 2017

Awesome, thanks! It's working now.

@superbock
Copy link
Collaborator Author

Thanks for testing :) Will fix it and (hopefully) release v0.15 very soon.

@superbock superbock requested review from fdlm and removed request for fdlm April 12, 2017 06:42
Copy link
Contributor

@fdlm fdlm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool beans! 👍
Just some minor comments....

docs/usage.rst Outdated

If no output directory is given, the program writes the output files to same
location as the audio files.
location as the the audio files.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo :)

@@ -48,38 +49,47 @@ class RNNBeatProcessor(SequentialProcessor):

"""

def __init__(self, post_processor=average_predictions, **kwargs):
def __init__(self, post_processor=average_predictions, online=False,
nn_files=None, **kwargs):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Is there any use case for the nn_files parameter?
  • Does it make sense to have the nn_files parameter if there is no frame_sizes parameter? As I see it, only neural networks trained on the given frame sizes (either for online or offline mode) would work.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Although you're right about which model can be used (only trained specifically for/with this settings), it can be quite useful to select only one or two of the trained models if processing power is limited.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand. But then this use case should be explained in the docs, I think it's not obvious from the beginning. Now that I've checked, the online parameter is also missing from the docstring of the class.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added docstrings and doctests.

@@ -92,6 +92,9 @@ def process(self, data, reset=True, **kwargs):
Network predictions for this data.

"""
# make data at least 2d (required by NN-layers)
if data.ndim < 2:
data = np.array(data, subok=True, copy=False, ndmin=2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just out of curiosity, is this better than np.atleast_2d?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is way faster on all machines I tested. I only spotted this when profiling the whole online beat tracking stuff, and it had a quite remarkable impact.

@superbock superbock merged commit 2a987a5 into master Apr 13, 2017
@superbock superbock deleted the online_beat_tracking branch April 13, 2017 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants