Batched matrix multiplication. #1261

FOsterfeld · 2023-11-08T13:36:03Z

split dimension is a batch dimension

Due Diligence

General:
- base branch must be main for new features, latest release branch (e.g. release/1.3.x) for bug fixes
- title of the PR is suitable to appear in the Release Notes
Implementation:
- unit tests: all split configurations tested
- unit tests: multiple dtypes tested
- documentation updated where needed

Description

Issue/s resolved: #

Changes proposed:

Type of change

Memory requirements

Performance

Does this change modify the behaviour of other functions? If so, which?

yes / no

split dimension is a batch dimension

ghost · 2023-11-08T13:37:51Z

👇 Click on the image for a new way to code review

Legend

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2023-11-08T14:23:07Z

Thank you for the PR!

codecov · 2023-11-08T15:04:00Z

Codecov Report

Attention: Patch coverage is 94.17040% with 13 lines in your changes missing coverage. Please review.

Project coverage is 91.93%. Comparing base (bc5ec47) to head (2719f4d).

Files	Patch %	Lines
heat/core/linalg/basics.py	94.17%	13 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1261      +/-   ##
==========================================
+ Coverage   91.87%   91.93%   +0.05%     
==========================================
  Files          80       80              
  Lines       11822    12041     +219     
==========================================
+ Hits        10862    11070     +208     
- Misses        960      971      +11

Flag	Coverage Δ
unit	`91.93% <94.17%> (+0.05%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mrfh92 · 2023-11-10T16:33:22Z

heat/core/linalg/basics.py

@@ -487,9 +487,12 @@ def matmul(a: DNDarray, b: DNDarray, allow_resplit: bool = False) -> DNDarray:
 sanitation.sanitize_in(a)
 sanitation.sanitize_in(b)

- if a.gshape[-1] != b.gshape[0]:
+ batch_dim = max(a.ndim, b.ndim) - 2
+ batched = batch_dim > 2


maybe > 0 instead of >2 or did I understand sth wrong?

Thanks, you're right, it should be 0.

mrfh92 · 2023-11-13T07:59:24Z

Sieht gut aus soweit 👍

Vorschläge für die weitere Arbeit:

Wenn du oben neben "codecov" (rotes X) auf Details und dann auf "View this Pull Request on Codecov" gehst, kannst du sehen, welche Zeilen deiner Änderungen nicht getestet werden bisher. Entsprechende Tests kannst du in heat/core/linalg/tests/test_basics.py unterbringen.
Mit diesem Fall wäre dann gebatchte Matrizenmultiplikation für den Fall abgedeckt, dass die LA-Dimensionen nicht gesplitted sind. Dies entspricht quasi dem Hinzufügen von Batchdimensionen zum Fall "2xsplit=None" im ursprünglichen Code.
Als nächstes könnten wir den Fall angehen, dass zu den anderen 8 (?) bestehenden Fällen Batch-dimensionen hinzugefügt werden; nun wäre allerdings die Batchdimension nicht gesplitted und dafür evtl. die LA-Dimension. Mit etwas Glück lässt sich das recht einfach aus dem bestehenden Code erzeugen, da ja PyTorch bereits gut mit Batches arbeiten kann.
Den Fall, dass einmal eine LA-Dimension gesplitted ist und einmal eine Batchdimension gesplitted ist, würde ich ausschließen und eine enstprechende Fehlermeldung ausgeben.

github-actions · 2023-11-21T09:47:53Z

Thank you for the PR!

github-actions · 2023-11-21T10:23:35Z

Thank you for the PR!

…nsion

github-actions · 2023-11-21T11:20:02Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2023-11-21T12:15:34Z

Thank you for the PR!

github-actions · 2023-11-21T14:09:39Z

Thank you for the PR!

github-actions · 2023-11-21T14:46:27Z

Thank you for the PR!

github-actions · 2023-11-21T15:08:15Z

Thank you for the PR!

github-actions · 2023-11-22T13:59:23Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-01-02T11:57:41Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-01-08T09:45:12Z

Thank you for the PR!

github-actions · 2024-01-09T15:52:21Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-01-22T12:21:22Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-02-01T08:26:09Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-02-05T08:29:25Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-02-06T15:57:52Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-02-14T07:13:11Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-02-20T07:05:23Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-02-29T16:59:21Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-03-07T11:10:12Z

Thank you for the PR!

…plit None

github-actions · 2024-03-19T10:21:26Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-06-03T08:12:38Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-06-20T23:18:24Z

Thank you for the PR!

FOsterfeld · 2024-06-20T23:19:28Z

Regarding the possible memory issues of #360, I benchmarked the maximal memory usage of matmul for two n x n matrices in the possible split cases.

first implementation of the minimal solution

67943f9

split dimension is a batch dimension

mrfh92 marked this pull request as draft November 8, 2023 13:37

FOsterfeld and others added 2 commits November 8, 2023 14:41

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

1c60823

…gebra_for_arrays_with_dimension_2_in_particular_matmul

access b.gshape[-2] only if input is not batched

0ddba6c

mrfh92 reviewed Nov 10, 2023

View reviewed changes

LScheib assigned LScheib and FOsterfeld and unassigned LScheib Nov 13, 2023

fixed batched condition

d60c0ca

throw a NotImplementedError for wrong split dimension on batched matmul

a60d3ac

FOsterfeld added 2 commits November 21, 2023 12:13

fixed dimension condition

e16366b

added test for batched matmul with split dimension being a batch dime…

7644dd4

…nsion

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

5d34282

…gebra_for_arrays_with_dimension_2_in_particular_matmul

FOsterfeld added 2 commits November 21, 2023 15:04

fixed condition for different batch dimensions

095ccc5

added some tests for correctly thrown errors

e55f8b8

fixed test for batched matmul on gpu

a9ae2bf

test for batched matmul on gpu

06913c7

remove unnecessary test with device=gpu

ba60c82

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

9733a28

…gebra_for_arrays_with_dimension_2_in_particular_matmul

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

0008a3f

…gebra_for_arrays_with_dimension_2_in_particular_matmul

added batched matmul tests for float values

18cdcf1

mrfh92 mentioned this pull request Jan 10, 2024

Batched Linalg beyond matmul #1320

Open

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

4e49aa5

…gebra_for_arrays_with_dimension_2_in_particular_matmul

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

bb0856b

…gebra_for_arrays_with_dimension_2_in_particular_matmul

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

0d37ff4

…gebra_for_arrays_with_dimension_2_in_particular_matmul

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

8531106

…gebra_for_arrays_with_dimension_2_in_particular_matmul

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

c911e45

…gebra_for_arrays_with_dimension_2_in_particular_matmul

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

5a2ad15

…gebra_for_arrays_with_dimension_2_in_particular_matmul

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

e804c2c

…gebra_for_arrays_with_dimension_2_in_particular_matmul

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

e5ff10b

…gebra_for_arrays_with_dimension_2_in_particular_matmul

improved exception throwing: error message when only one matrix has s…

da9b0e3

…plit None

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

f3e0ced

…gebra_for_arrays_with_dimension_2_in_particular_matmul

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

2719f4d

…gebra_for_arrays_with_dimension_2_in_particular_matmul

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Batched matrix multiplication. #1261

Batched matrix multiplication. #1261

FOsterfeld commented Nov 8, 2023

ghost commented Nov 8, 2023 •

edited by ghost

Loading

Legend

github-actions bot commented Nov 8, 2023

codecov bot commented Nov 8, 2023 •

edited

Loading

mrfh92 Nov 10, 2023 •

edited

Loading

FOsterfeld Nov 21, 2023

mrfh92 commented Nov 13, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 22, 2023

github-actions bot commented Jan 2, 2024

github-actions bot commented Jan 8, 2024

github-actions bot commented Jan 9, 2024

github-actions bot commented Jan 22, 2024

github-actions bot commented Feb 1, 2024

github-actions bot commented Feb 5, 2024

github-actions bot commented Feb 6, 2024

github-actions bot commented Feb 14, 2024

github-actions bot commented Feb 20, 2024

github-actions bot commented Feb 29, 2024

github-actions bot commented Mar 7, 2024

github-actions bot commented Mar 19, 2024

github-actions bot commented Jun 3, 2024

github-actions bot commented Jun 20, 2024

FOsterfeld commented Jun 20, 2024 •

edited

Loading

Batched matrix multiplication. #1261

Are you sure you want to change the base?

Batched matrix multiplication. #1261

Conversation

FOsterfeld commented Nov 8, 2023

Due Diligence

Description

Changes proposed:

Type of change

Memory requirements

Performance

Does this change modify the behaviour of other functions? If so, which?

ghost commented Nov 8, 2023 • edited by ghost Loading

Legend

github-actions bot commented Nov 8, 2023

codecov bot commented Nov 8, 2023 • edited Loading

Codecov Report

mrfh92 Nov 10, 2023 • edited Loading

Choose a reason for hiding this comment

FOsterfeld Nov 21, 2023

Choose a reason for hiding this comment

mrfh92 commented Nov 13, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 22, 2023

github-actions bot commented Jan 2, 2024

github-actions bot commented Jan 8, 2024

github-actions bot commented Jan 9, 2024

github-actions bot commented Jan 22, 2024

github-actions bot commented Feb 1, 2024

github-actions bot commented Feb 5, 2024

github-actions bot commented Feb 6, 2024

github-actions bot commented Feb 14, 2024

github-actions bot commented Feb 20, 2024

github-actions bot commented Feb 29, 2024

github-actions bot commented Mar 7, 2024

github-actions bot commented Mar 19, 2024

github-actions bot commented Jun 3, 2024

github-actions bot commented Jun 20, 2024

FOsterfeld commented Jun 20, 2024 • edited Loading

ghost commented Nov 8, 2023 •

edited by ghost

Loading

codecov bot commented Nov 8, 2023 •

edited

Loading

mrfh92 Nov 10, 2023 •

edited

Loading

FOsterfeld commented Jun 20, 2024 •

edited

Loading