Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set UCX_CUDA_COPY_MAX_REG_RATIO=1.0 when GPUs permit #824

Merged
merged 3 commits into from
Jan 4, 2022
Merged

Set UCX_CUDA_COPY_MAX_REG_RATIO=1.0 when GPUs permit #824

merged 3 commits into from
Jan 4, 2022

Commits on Jan 3, 2022

  1. Set UCX_CUDA_COPY_MAX_REG_RATIO=1.0 when GPU permits 

    This is necessary to maintain good performance on GPUs with large BAR1
size (greater or equal to the total GPU memory size). Starting with UCX
1.12 this value is set to 0.1 by default to ensure registration doesn't
fail with GPUs that have a BAR1 size lower than the total GPU memory
size (e.g., T4).

Currently we're setting to 1.0 when all GPUs have a large BAR1, in the
future it may be useful to compute the real BAR1/TotalGPUMemory ratio to
set this number. This is also a global setting, for fine-grained
settings the application should take care of changing this value
appropriately as UCX-Py doesn't know which GPUs are going to be used by
the application at import time.
    @pentschev
    pentschev committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    9097664 View commit details
    Browse the repository at this point in the history

  2. Standardize os.environ checks

    @pentschev
    pentschev committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    a5ce736 View commit details
    Browse the repository at this point in the history

  3. Setup logger before environment variables and log variables set

    @pentschev
    pentschev committed Jan 3, 2022
    Configuration menu
    Copy the full SHA
    2dfb241 View commit details
    Browse the repository at this point in the history