[#944] Improve performance #2108

SkyBlaise99 · 2024-02-06T08:11:05Z

Part of #944

Proposed commit message

Currently, the authorship credit analysis process exceeds an hour in
duration, which is significantly longer than the mere 5 minutes required
when the feature is deactivated.

Let's speed up the performance by implementing a caching mechanism and
refining the dynamic programming algorithm utilized for computing the
Levenshtein distance.

Other information

Statistics obtained by running on CS2103/T AY22/23 Semester 2 student repositories.

Forked here and some statistics recorded on the Github Actions.

Case	Runtime
Base Case	1h 30min
Cache	15min
Lev Distance	1h 8min
Cache + Lev Distance	13 min

Currently, users are unable to select a zoom range that includes the until date. This results in misleading data being presented to users.

…ense#2041) Chrome bug is causing cypress to fail to open a browser on Github Actions, causing frontend tests and CI to fail. Upgrading cypress to greater than 12.15.0 will fix this issue. Let's upgrade cypress to fix the failing CI.

Currently, there is still some JavaScript code which remains unmigrated. This allows for type unsafe code to be written, potentially resulting in unintended behavior. Let's migrate the rest of the JavaScript code to TypeScript code to facilitate future changes to the code.

…posense#2040) Currently, there is still some JavaScript code which remains unmigrated. This allows for type unsafe code to be written, potentially resulting in unintended behavior. Let's migrate the rest of the JavaScript code to TypeScript code to facilitate future changes to the code.

Currently, Cypress zoom feature tests are failing due to a recent change in behavior caused by a bug fix. With the tests failing, we are unable to detect any future regressions. Let's update the Cypress tests to test for the new intended behavior.

…#2043) Currently, there is still some JavaScript code which remains unmigrated. This allows for type unsafe code to be written, potentially resulting in unintended behavior. Let's migrate random-color-generator.js JavaScript code to TypeScript code to facilitate future changes to the code.

…sense#2036) Currently, there is still some JavaScript code which remains unmigrated. This allows for type unsafe code to be written, potentially resulting in unintended behavior. Let's migrate the rest of the JavaScript code to TypeScript code to facilitate future changes to the code.

Currently, there is still some JavaScript code which remains unmigrated. This allows for type unsafe code to be written, potentially resulting in unintended behavior. Let's migrate the rest of the JavaScript code to TypeScript code to facilitate future changes to the code.

Bumps [zod](https://github.com/colinhacks/zod) from 3.20.6 to 3.22.3. - [Release notes](https://github.com/colinhacks/zod/releases) - [Changelog](https://github.com/colinhacks/zod/blob/master/CHANGELOG.md) - [Commits](colinhacks/zod@v3.20.6...v3.22.3) --- updated-dependencies: - dependency-name: zod dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [@cypress/request](https://github.com/cypress-io/request) to 3.0.1 and updates ancestor dependency [cypress](https://github.com/cypress-io/cypress). These dependencies need to be updated together. Updates `@cypress/request` from 2.88.12 to 3.0.1 - [Release notes](https://github.com/cypress-io/request/releases) - [Changelog](https://github.com/cypress-io/request/blob/master/CHANGELOG.md) - [Commits](cypress-io/request@v2.88.12...v3.0.1) Updates `cypress` from 12.17.4 to 13.3.0 - [Release notes](https://github.com/cypress-io/cypress/releases) - [Changelog](https://github.com/cypress-io/cypress/blob/develop/CHANGELOG.md) - [Commits](cypress-io/cypress@v12.17.4...v13.3.0) --- updated-dependencies: - dependency-name: "@cypress/request" dependency-type: indirect - dependency-name: cypress dependency-type: direct:development ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Currently, there is still some JavaScript code which remains unmigrated. This allows for type unsafe code to be written, potentially resulting in unintended behavior. Let's migrate the rest of the JavaScript code to TypeScript code to facilitate future changes to the code.

author

…-git-log

Currently, when granularity is set to day or week, clicking on a ramp will open up a zoom view where commit messages are not being displayed and sorting by insertions does not result in any sorting. Let's fix the unintended behaviour of the zoom view.

Currently, there is still some JavaScript code which remains unmigrated. This allows for type unsafe code to be written, potentially resulting in unintended behavior. Let's migrate repo-sorter.js to TypeScript code to facilitate future changes to the code.

Currently, there is still some JavaScript code which remains unmigrated. This allows for type unsafe code to be written, potentially resulting in unintended behavior. Let's migrate safari_date.js to TypeScript code to facilitate future changes to the code.

Currently, frontend linter is failing due to lint scripts checking javascript files, the last of which has been removed in PR reposense#2053. Lets update the lint command to exclude javascript files front the check.

SkyBlaise99 · 2024-02-06T09:08:08Z

@damithc Hi prof, the part related to lev dist is here.

damithc · 2024-02-06T09:33:13Z

@damithc Hi prof, the part related to lev dist is here.

@SkyBlaise99 Got it. The increase in complexity (in that method) is not that bad.

MarcusTXK

LGTM, code is well commented and clean. Great job with the improvement in the performance

gok99

LGTM - haven't reasoned through the details of the algorithm updates, but I'll trust the comprehensiveness of the tests.

I'm curious what the memory consumption hit is like if you happened to measure this too - the intelliJ profiler should be able to provide some information.

SkyBlaise99 · 2024-03-02T08:48:37Z

I'm curious what the memory consumption hit is like if you happened to measure this too - the intelliJ profiler should be able to provide some information.

I'm not too familiar with intellij profiler. I tried using it initially but due to the long runtime on local env, I stopped using it. Instead I used this telemetry which also measures the ram usage, but it shows a rough scale and not the exact number.

Currently I'm looking at 2GB vs 1.5GB, a 0.5GB increase in ram usage when caching is enabled. As compared to 7GB limit for github actions, this should still be okay.

github-actions · 2024-03-03T05:01:24Z

The following links are for previewing this pull request:

Dashboard Preview: https://dashboard-2108-pr-reposense-reposense.surge.sh
Docs Preview: https://docs-2108-pr-reposense-reposense.surge.sh

SkyBlaise99 and others added 30 commits September 12, 2023 21:45

use AY2223S2 tp repo

93c0f75

override equals for candidate line

613faeb

prevent frontend from installing everytime

f45f81f

Merge branch '944-test-base' into 944-test-cache

7a04297

prevent build frontend instead

57018b2

Merge branch '944-test-base' into 944-test-cache

48d2c8f

[reposense#2027] Fix date range bug (reposense#2034)

075d34d

Currently, users are unable to select a zoom range that includes the until date. This results in misleading data being presented to users.

cache git diff results

595b6fd

compute more then cache

65acc53

fix bug in analysis

7736d6f

rename key

6767123

add cache for git log

857201f

Merge branch 'master' into 944-analyze-authorship

174ecc5

Give partial credit if annotated author is not the same as the blame

e200e5e

author

Merge branch '944-analyze-authorship' into 944-test-cache-git-diff-v2…

65d8e99

…-git-log

Remove frontend JS lint (reposense#2063)

056fa5f

Currently, frontend linter is failing due to lint scripts checking javascript files, the last of which has been removed in PR reposense#2053. Lets update the lint command to exclude javascript files front the check.

use full and partial credit color

bdeb15a

SkyBlaise99 added 10 commits February 4, 2024 18:37

reduce space complexity down to O(min(s, t))

3f933e4

reduce time complexity

bbb2dc9

add early termination

ef2d67d

early termination if limit is reached

fb95942

early termination

f1fb667

add cache for git log and git diff

8f682a8

reduce space complexity to O(min(s, t))

5e7e1e6

add several early termination

11f16e2

Merge branch '944-cache' into 944-improve-performance

ba457b3

Merge branch '944-lev-dist' into 944-improve-performance

551f5d5

SkyBlaise99 changed the title ~~944 improve performance~~ [#944] Improve Performance Feb 6, 2024

SkyBlaise99 changed the title ~~[#944] Improve Performance~~ [#944] Improve performance Feb 6, 2024

fix checkstyle and update comments

9e9f961

SkyBlaise99 marked this pull request as ready for review February 6, 2024 12:09

ckcherry23 requested a review from a team February 6, 2024 14:55

ckcherry23 added the s.ToReview label Feb 6, 2024

ckcherry23 requested a review from a team February 19, 2024 06:25

MarcusTXK approved these changes Mar 2, 2024

View reviewed changes

MarcusTXK requested a review from a team March 2, 2024 05:47

gok99 approved these changes Mar 2, 2024

View reviewed changes

gok99 added s.ToMerge and removed s.ToReview labels Mar 2, 2024

gok99 merged commit fdd2dae into reposense:944-analyze-authorship Mar 3, 2024
10 checks passed

gok99 mentioned this pull request Mar 3, 2024

[#944] Add originality threshold flag #2122

Merged

SkyBlaise99 deleted the 944-improve-performance branch March 3, 2024 07:00

SkyBlaise99 restored the 944-improve-performance branch March 3, 2024 07:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[#944] Improve performance #2108

[#944] Improve performance #2108

SkyBlaise99 commented Feb 6, 2024 •

edited

Loading

SkyBlaise99 commented Feb 6, 2024

damithc commented Feb 6, 2024

MarcusTXK left a comment

gok99 left a comment •

edited

Loading

SkyBlaise99 commented Mar 2, 2024 •

edited

Loading

github-actions bot commented Mar 3, 2024

[#944] Improve performance #2108

[#944] Improve performance #2108

Conversation

SkyBlaise99 commented Feb 6, 2024 • edited Loading

Proposed commit message

Other information

SkyBlaise99 commented Feb 6, 2024

damithc commented Feb 6, 2024

MarcusTXK left a comment

Choose a reason for hiding this comment

gok99 left a comment • edited Loading

Choose a reason for hiding this comment

SkyBlaise99 commented Mar 2, 2024 • edited Loading

github-actions bot commented Mar 3, 2024

SkyBlaise99 commented Feb 6, 2024 •

edited

Loading

gok99 left a comment •

edited

Loading

SkyBlaise99 commented Mar 2, 2024 •

edited

Loading