Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#2161] One-Stop Config File for Code Portfolio #2172

Closed
wants to merge 30 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
2db2ac0
Update with report config YAML files
asdfghjkxd Mar 25, 2024
202d247
Undo SummaryJson changes
asdfghjkxd Mar 25, 2024
f64f6ed
Update ReportConfiguration to support branches
asdfghjkxd Mar 27, 2024
356299a
Update Report Configuration docs
asdfghjkxd Mar 27, 2024
ffd4912
Merge branch 'master' into 2161-code-portfolio
asdfghjkxd Mar 27, 2024
2cbb662
Update YAML format
asdfghjkxd Mar 28, 2024
9190fb0
Fix environmental error
asdfghjkxd Apr 1, 2024
b611f29
Implement Markdown Parser
asdfghjkxd Apr 2, 2024
6573637
Fix test cases
asdfghjkxd Apr 2, 2024
6c5e401
Add blurbs on frontend (#2186)
sopa301 Apr 4, 2024
4259db3
Merge branch 'master' into 2161-code-portfolio
asdfghjkxd Apr 4, 2024
43f1c14
Add tests for BlurbMap and ReportConfiguration
asdfghjkxd Apr 4, 2024
d5fd31a
Remove manual testing artefact
sopa301 Apr 4, 2024
6e58493
Upgrade checkstyle and fix checkstyle errors
asdfghjkxd Apr 5, 2024
7924629
Revert "Upgrade checkstyle and fix checkstyle errors"
asdfghjkxd Apr 5, 2024
cbf622d
Update regex
asdfghjkxd Apr 8, 2024
fa95f75
Remove unused methods
asdfghjkxd Apr 8, 2024
4299ff2
Update Regex to accept 0 or more characters
asdfghjkxd Apr 8, 2024
1145f8e
Modify css
sopa301 Apr 8, 2024
7c94fea
Merge branch '2161-code-portfolio' of https://github.com/reposense/Re…
sopa301 Apr 8, 2024
6ee3249
Update Yaml file format
asdfghjkxd Apr 8, 2024
48c873b
Add blurbs to frontend files
sopa301 Apr 8, 2024
50f6ec6
Fix style
sopa301 Apr 8, 2024
941f420
Fix style
sopa301 Apr 8, 2024
6dd383b
Move blurb parsing into ArgsParser
asdfghjkxd Apr 11, 2024
d628e85
Merge branch '2161-code-portfolio' of https://github.com/reposense/Re…
asdfghjkxd Apr 11, 2024
68ea84e
Fix tests
sopa301 Apr 11, 2024
28d9c84
Change config files
sopa301 Apr 11, 2024
83329cf
Update blurb map parsing
asdfghjkxd Apr 11, 2024
f70ead4
Merge branch '2161-code-portfolio' of https://github.com/reposense/Re…
asdfghjkxd Apr 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ mainClassName = 'reposense.RepoSense'

node.download = false // The Liferay Node Gradle Plugin will use the system PATH to find the Node/npm executable.

sourceCompatibility = JavaVersion.VERSION_1_8
targetCompatibility = JavaVersion.VERSION_1_8
sourceCompatibility = JavaVersion.VERSION_11
targetCompatibility = JavaVersion.VERSION_11

repositories {
mavenCentral()
Expand All @@ -38,6 +38,8 @@ dependencies {
implementation group: 'org.apache.ant', name: 'ant', version: '1.10.12'
implementation group: 'org.apache.commons', name: 'commons-csv', version: '1.9.0'
implementation group: 'org.fusesource.jansi', name: 'jansi', version: '2.4.0'
implementation group: 'com.fasterxml.jackson.datatype', name: 'jackson-datatype-jsr310', version: '2.17.0'
implementation group: 'com.fasterxml.jackson.dataformat', name: 'jackson-dataformat-yaml', version: '2.17.0'

testImplementation group: 'org.junit.jupiter', name: 'junit-jupiter-api', version: jUnitVersion
testImplementation group: 'org.junit.jupiter', name: 'junit-jupiter-engine', version: jUnitVersion
Expand Down
2 changes: 2 additions & 0 deletions config/blurbs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
https://github.com/reposense/testrepo-Alpha/tree/master
Master branch of testrepo-Alpha
Empty file modified config/checks/check-eof-newline.sh
100644 → 100755
Empty file.
Empty file modified config/checks/check-line-endings.sh
100644 → 100755
Empty file.
Empty file modified config/checks/check-trailing-whitespace.sh
100644 → 100755
Empty file.
43 changes: 43 additions & 0 deletions config/report-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
title: RepoSense Report
group-details:
- repo: https://github.com/user/repo/tree/fake-branch
groups:
- group-name: code
globs:
- "**.java"
- group-name: tests
globs:
- "src/test**"
- group-name: docs
globs:
- "docs**"
- "**.adoc"
- "**.md"
repos:
- repo: https://github.com/user/repo/tree/fake-branch
author-emails:
- [email protected]
- [email protected]
- [email protected]
author-git-host-id: johnDoe
author-display-name: John Doe
author-git-author-name: my home PC
branches:
- branch: main
file-formats:
- override:java
- md
- fxml
ignore-glob-list:
- "docs**"
ignore-standalone-config: true
ignore-commits-list:
- 2fb6b9b2dd9fa40bf0f9815da2cb0ae8731436c7
- c5a6dc774e22099cd9ddeb0faff1e75f9cf4f151
- cd7f610e0becbdf331d5231887d8010a689f87c7
- 768015345e70f06add2a8b7d1f901dc07bf70582
ignore-authors-list:
- author1
- author2
is-shallow-cloning: true
is-find-previous-authors: false
16 changes: 13 additions & 3 deletions docs/ug/configFiles.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,12 +107,22 @@ e.g.: `example.java` in `example-repo` can either be in the `test` group or the

<!-- ==================================================================================================== -->

## `report-config.json`
## `report-config.yaml`

You can optionally use `report-config.json` to customize report generation by providing the following information. ([example](report-config.json))
You can also optionally use a `report-config.json` file to quickly define the repository information for the repositories you are interested in tracking and generate your very own code portfolio.

View this [example](report-config.yaml) for a better understanding of what repository information is required.

**Fields to provide**:
* `title`: Title of the generated report, which is also the title of the deployed dashboard. Default: "RepoSense Report"
* `repoUrl`: The URL to your repository of interest
* `reportTitle`: Title of the generated report, which is also the title of the deployed dashboard. Default: "RepoSense Report"
* `authorDisplayName`: Name of the author to track in the repository.
* `authorGithubId`: The GitHub username of the author to track in the repository.
* `branches`: A list of branches with their associated blurbs.
* `name`: Name of the branch to track
* `blurb`: Blurb to include with the branch
* `startDate`: The start date of analysis for RepoSense. Default: "2020-01-01"
* `endDate`: The end date of analysis for RepoSense. Default: "9999-12-32"

<!-- ==================================================================================================== -->

Expand Down
3 changes: 0 additions & 3 deletions docs/ug/report-config.json

This file was deleted.

11 changes: 11 additions & 0 deletions docs/ug/report-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
repoUrl: github.com/user/repo
reportTitle: RepoSense Report
authorDisplayName: Sample Author
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@damithc Sure thing, will remove these from the YAML file and other associated classes in a future commit.

authorGithubId: Sample Author Github ID
branches:
- name: Branch 1
blurb: Blurb 1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blurb should allow a few paragraph of markdown text. So, yml might not be the right format for this config file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@damithc There are ways to incorporate multi-line strings into YAML files here, we could perhaps include this into the docs to let users know how they can tailor the blurbs to suit their use cases (multi-paragraph vs long single paragraph text)

- name: Branch 2
blurb: Blurb 2
startDate: 2020-01-01
endDate: 9999-12-31
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #2161 (comment) for an example that seems intuitive from the user POV

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@damithc Just wanted to clarify, do you mean that we should include the configs within the markdown files itself, and have the backend parse the markdown files and then generate the reports, or that the overall order/style/keys in the YAML file should match what was used in the example provided?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asdfghjkxd I wasn't specifically using markdown either. It is just a custom format that makes sense from the user's point of view. I'm fine if something close can be achieved by YAML or some other commonly-used format, provided it makes sense from the user (i.e., the user shouldn't have to do a lot of extra work just to conform to the file format)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@damithc I see, sure thing, we will try to design a user-friendly and convenient config file format and update this PR and parent issue with any new findings/improvements!

5 changes: 5 additions & 0 deletions frontend/cypress/config/blurbs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
https://github.com/reposense/RepoSense/tree/cypress
first blurb
<!--repo-->------------------------------------
https://gitlab.com/reposense/testrepo-gitlab/-/tree/main
unseen blurb
1 change: 1 addition & 0 deletions frontend/cypress/config/report-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
title: RepoSense Test Report
12 changes: 12 additions & 0 deletions frontend/cypress/tests/chartView/chartView_blurbs.cy.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
describe('blurbs', () => {
it('shows blurbs', () => {
cy.get('.markdown.blurb')
.first()
.should('contain', 'first blurb');
});

it('has the correct number of valid blurbs', () => {
cy.get('.markdown.blurb')
.should('have.length', 1);
});
});
2 changes: 2 additions & 0 deletions frontend/src/app.vue
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ const app = defineComponent({
reportGenerationTime,
errorMessages,
names,
blurbMap,
} = summary;
this.creationDate = creationDate;
this.reportGenerationTime = reportGenerationTime;
Expand All @@ -125,6 +126,7 @@ const app = defineComponent({
this.getUsers();
this.renderTabHash();
this.userUpdated = true;
this.$store.commit('setBlurbMap', blurbMap);
} catch (error) {
window.alert(error);
} finally {
Expand Down
30 changes: 30 additions & 0 deletions frontend/src/components/c-markdown-chunk.vue
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
<template lang="pug">
.markdown(v-html="parsedHtml", v-if="markdownText != ''")
</template>

<script lang="ts">
import { defineComponent } from 'vue';
import MarkdownIt from 'markdown-it';

export default defineComponent({
props: {
markdownText: {
type: String,
required: true,
},
},
computed: {
parsedHtml() {
const md = new MarkdownIt({ html: true });
return md.render(this.markdownText);
},
},
});
</script>

<style lang="scss" scoped>
.markdown {
overflow-x: auto;
padding: 0 1.5rem;
}
</style>
39 changes: 37 additions & 2 deletions frontend/src/components/c-summary-charts.vue
Original file line number Diff line number Diff line change
Expand Up @@ -134,11 +134,18 @@
)
font-awesome-icon.icon-button(icon="clipboard")
span.tooltip-text(v-bind:ref="`summary-charts-${i}-copy-iframe`") Click to copy iframe link for group

.tooltip.summary-chart__title--percentile(
v-if="sortGroupSelection.includes('totalCommits')"
v-if="sortGroupSelection.includes('totalCommits')"
) {{ getPercentile(i) }} %&nbsp
span.tooltip-text.right-aligned {{ getPercentileExplanation(i) }}

.blurbWrapper(
v-if="filterGroupSelection === 'groupByRepos'",
)
c-markdown-chunk.blurb(
v-bind:markdown-text="getBlurb(repo[0])"
)

.summary-charts__fileType--breakdown(v-if="filterBreakdown")
template(v-if="filterGroupSelection !== 'groupByNone'")
.summary-charts__fileType--breakdown__legend(
Expand Down Expand Up @@ -298,6 +305,7 @@ import brokenLinkDisabler from '../mixin/brokenLinkMixin';
import tooltipPositioner from '../mixin/dynamicTooltipMixin';
import cRamp from './c-ramp.vue';
import cStackedBarChart from './c-stacked-bar-chart.vue';
import cMarkdownChunk from './c-markdown-chunk.vue';
import { Bar, Repo, User } from '../types/types';
import { FilterGroupSelection, FilterTimeFrame, SortGroupSelection } from '../types/summary';
import { StoreState, ZoomInfo } from '../types/vuex.d';
Expand All @@ -308,6 +316,7 @@ export default defineComponent({
components: {
cRamp,
cStackedBarChart,
cMarkdownChunk,
},
mixins: [brokenLinkDisabler, tooltipPositioner],
props: {
Expand Down Expand Up @@ -875,6 +884,32 @@ export default defineComponent({

return totalContribution / totalCommits;
},

getBlurb(repo: User): string {
const link = this.getRepoLink(repo);
if (!link) {
return '';
}
const blurb: string | undefined = this.$store.state.blurbMap[link];
if (!blurb) {
return '';
}
return blurb;
},
},
});
</script>

<style lang="scss" scoped>
@import '../styles/_colors.scss';

.blurbWrapper {
padding-bottom: 5px;
.blurb {
background-color: #fafafa;
overflow-y: hidden;
// This is needed because the parent summary-wrapper center aligns everything
text-align: initial;
}
}
</style>
13 changes: 7 additions & 6 deletions frontend/src/components/c-title.vue
Original file line number Diff line number Diff line change
@@ -1,12 +1,16 @@
<template lang="pug">
.title(v-html="markdownText", v-if="markdownText != ''")
.title
c-markdown-chunk(v-bind:markdown-text="markdownText")
</template>

<script lang="ts">
import MarkdownIt from 'markdown-it';
import { defineComponent } from 'vue';
import cMarkdownChunk from './c-markdown-chunk.vue';

export default defineComponent({
components: {
cMarkdownChunk,
},
data() {
return {
markdownText: '',
Expand All @@ -19,8 +23,7 @@ export default defineComponent({
}
return response.text();
}).then((text) => {
const md = new MarkdownIt({ html: true });
this.markdownText = md.render(text);
this.markdownText = text;
}).catch((error) => {
this.markdownText = (error as Error).toString();
});
Expand All @@ -30,8 +33,6 @@ export default defineComponent({

<style lang="scss" scoped>
.title {
overflow-x: auto;
padding: 0 1.5rem;
// This is needed because the parent summary-wrapper center aligns everything
text-align: initial;
}
Expand Down
4 changes: 4 additions & 0 deletions frontend/src/store/store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ export default createStore<StoreState>({
loadingOverlayCount: 0,
loadingOverlayMessage: '',
isTabActive: true,
blurbMap: {},
} as StoreState,
mutations: {
updateTabZoomInfo(state: StoreState, info: ZoomInfo) {
Expand Down Expand Up @@ -82,6 +83,9 @@ export default createStore<StoreState>({
file.wasCodeLoaded = file.wasCodeLoaded || file.active;
});
},
setBlurbMap(state: StoreState, blurbMap: { [key: string]: string }) {
state.blurbMap = blurbMap;
},
},
actions: {
// Actions are called with dispatch
Expand Down
1 change: 1 addition & 0 deletions frontend/src/types/vuex.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ interface StoreState {
loadingOverlayCount: number;
loadingOverlayMessage: string;
isTabActive: boolean;
blurbMap: { [key: string]: string };
}

declare module '@vue/runtime-core' {
Expand Down
1 change: 1 addition & 0 deletions frontend/src/types/window.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ interface Api {
reportGenerationTime: string;
errorMessages: { [key: string]: ErrorMessage };
names: string[];
blurbMap: { [key: string]: string };
} | null>;
loadCommits: (repoName: string) => Promise<User[]>;
loadAuthorship: (repoName: string) => Promise<AuthorshipSchema>;
Expand Down
3 changes: 3 additions & 0 deletions frontend/src/types/zod/summary-type.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@ export const summarySchema = z.object({
isSinceDateProvided: z.boolean(),
isUntilDateProvided: z.boolean(),
supportedDomainUrlMap: supportedDomainUrlMapSchema,
blurbs: z.object({
urlBlurbMap: z.record(z.string(), z.string()),
}),
});

// Export typescript types
Expand Down
3 changes: 3 additions & 0 deletions frontend/src/utils/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -232,11 +232,14 @@ window.api = {
window.REPOS[repoName] = repo;
names.push(repoName);
});

const blurbMap: { [key: string]: string } = data.blurbs.urlBlurbMap;
return {
creationDate: reportGeneratedTime,
reportGenerationTime,
errorMessages,
names,
blurbMap,
};
},

Expand Down
13 changes: 11 additions & 2 deletions src/main/java/reposense/RepoSense.java
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,15 @@

import net.sourceforge.argparse4j.helper.HelpScreenException;
import reposense.git.GitConfig;
import reposense.model.BlurbMap;
import reposense.model.CliArguments;
import reposense.model.RepoConfiguration;
import reposense.model.ReportConfiguration;
import reposense.model.RunConfigurationDecider;
import reposense.model.reportconfig.ReportConfiguration;
import reposense.parser.ArgsParser;
import reposense.parser.exceptions.InvalidCsvException;
import reposense.parser.exceptions.InvalidHeaderException;
import reposense.parser.exceptions.InvalidMarkdownException;
import reposense.parser.exceptions.ParseException;
import reposense.report.ReportGenerator;
import reposense.system.LogsManager;
Expand All @@ -43,6 +45,7 @@ public static void main(String[] args) {
CliArguments cliArguments = ArgsParser.parse(args);
List<RepoConfiguration> configs = null;
ReportConfiguration reportConfig = new ReportConfiguration();
BlurbMap blurbMap = new BlurbMap();

if (cliArguments.isViewModeOnly()) {
ReportServer.startServer(SERVER_PORT_NUMBER, cliArguments.getReportDirectoryPath().toAbsolutePath());
Expand All @@ -51,6 +54,7 @@ public static void main(String[] args) {

configs = RunConfigurationDecider.getRunConfiguration(cliArguments).getRepoConfigurations();
reportConfig = cliArguments.getReportConfiguration();
blurbMap = cliArguments.getBlurbMap();

RepoConfiguration.setFormatsToRepoConfigs(configs, cliArguments.getFormats());
RepoConfiguration.setDatesToRepoConfigs(configs, cliArguments.getSinceDate(), cliArguments.getUntilDate());
Expand Down Expand Up @@ -79,7 +83,9 @@ public static void main(String[] args) {
cliArguments.getSinceDate(), cliArguments.getUntilDate(),
cliArguments.isSinceDateProvided(), cliArguments.isUntilDateProvided(),
cliArguments.getNumCloningThreads(), cliArguments.getNumAnalysisThreads(),
TimeUtil::getElapsedTime, cliArguments.getZoneId(), cliArguments.isFreshClonePerformed());
TimeUtil::getElapsedTime, cliArguments.getZoneId(), cliArguments.isFreshClonePerformed(),
blurbMap
);

FileUtil.zipFoldersAndFiles(reportFoldersAndFiles, cliArguments.getOutputFilePath().toAbsolutePath(),
".json");
Expand All @@ -96,6 +102,9 @@ public static void main(String[] args) {
logger.log(Level.WARNING, e.getMessage(), e);
} catch (HelpScreenException e) {
// help message was printed by the ArgumentParser; it is safe to exit.
} catch (InvalidMarkdownException ex) {
logger.log(Level.SEVERE, ex.getMessage(), ex);
System.exit(1);
}

LogsManager.moveLogFileToOutputFolder();
Expand Down
Loading
Loading