Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CUDA][AMP] Size-1 (scalar) norms are broken on CUDA + AMP following #122143 #125174

Closed
eqy opened this issue Apr 29, 2024 · 0 comments
Closed

[CUDA][AMP] Size-1 (scalar) norms are broken on CUDA + AMP following #122143 #125174

eqy opened this issue Apr 29, 2024 · 0 comments
Labels
module: amp (automated mixed precision) autocast module: cuda Related to torch.cuda, and CUDA support in general module: norms and normalization triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@eqy
Copy link
Collaborator

eqy commented Apr 29, 2024 •
edited by pytorch-bot bot

🐛 Describe the bug

A corner case of AMP + norm seems broken after #122143

import torch

inp = torch.randn(1, device='cuda', dtype=torch.half)
with torch.autocast('cuda', dtype=torch.float16):
    torch.linalg.vector_norm(inp)

CC @Aidyn-A who is working on a fix
@min-jean-cho
@nWEIdia @tinglvv

Versions

Current nightlies

cc @ptrblck @mcarilli @leslie-fang-intel @jgong5
@eqy eqy added module: cuda Related to torch.cuda, and CUDA support in general module: amp (automated mixed precision) autocast module: norms and normalization labels Apr 29, 2024
@eqy eqy changed the title [CUDA][AMP] Size-1 (scalar) norms are broken on CUDA following #122143 [CUDA][AMP] Size-1 (scalar) norms are broken on CUDA + AMP following #122143 Apr 29, 2024
@Aidyn-A Aidyn-A mentioned this issue Apr 29, 2024
Closed
@cpuhrsch cpuhrsch added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Apr 30, 2024
andoorve pushed a commit to andoorve/pytorch that referenced this issue May 1, 2024
@Aidyn-A @andoorve
[ATen][CUDA][AMP] Fix dtype mismatch in linalg_vector_norm (pytorch#1…
141875e 
…25175)

Fixes pytorch#125174

Pull Request resolved: pytorch#125175
Approved by: https://github.com/eqy, https://github.com/lezcano
pytorch-bot bot pushed a commit that referenced this issue May 3, 2024
@Aidyn-A @petrex
[ATen][CUDA][AMP] Fix dtype mismatch in linalg_vector_norm (#125175)
4ff8b10 
Fixes #125174

Pull Request resolved: #125175
Approved by: https://github.com/eqy, https://github.com/lezcano
@ezhang887 ezhang887 mentioned this issue May 6, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
module: amp (automated mixed precision) autocast module: cuda Related to torch.cuda, and CUDA support in general module: norms and normalization triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[ATen][CUDA][AMP] Fix dtype mismatch in linalg_vector_norm Aidyn-A/pytorch
2 participants
@cpuhrsch @eqy