Repository navigation
Implement report.md formatting #148
Description
As user whose trying out Code PushUp for the first time, I want an easy to read (i.e. not raw JSON) report after I've succesfully ran the CLI for the first time (based on getting started docs).
Based on discussion in #93
General notes
- scores (0-1) formatted as
Math.round(score * 100)(0-100, only integers) - links to headings have to slugify the heading text
- everything that's in
<details>(including<summary>) must use HTML formatting, Markdown won't be rendered - score colours:
- category and groups scores are annotated with 🟢, 🟡, or 🔴
- audit value and score are annotated with 🟩, 🟨, or 🟥
- determined by score value:
score < 0.5=> red0.5 <= score < 0.9=> yellowscore >= 0.9=> green
Structure
- heading 1: "Code PushUp report"
- intro section
- Markdown table with category scores
- Category cells contain title with link to category detail
- Audits cells should number of referenced audits (from category refs, ref to group expanded to number of audits)
- heading 2: "Categories"
- categories section - lists all categories
- category detail:
- heading 3: category title
- score
- unordered list based on category refs (regardless of weight)
- referenced audits:
- colour based on audit's score
- audit title with link to audit detail
- plugin title in brackets
- display value in bold - if missing, show raw value instead
- referenced groups:
- colour based on group's score
- group title with plugin title in brackets
- sub-list with all audits referenced by group
- audit's formatting is the same as for those referenced by category directly (see above) - only difference is plugin title is omitted
- referenced audits:
- category detail:
- heading 2: "Audits"
- audits section - lists all audits from all plugins
- audit detail:
- heading 3: audit title with plugin in brackets
- result summary: colour from score, display/raw value in bold, score in brackets
- if audit has details with issues, result used as summary for expandable details section:
- heading 4: "Issues"
- table which list all issues (we only have source code issues at present)
- severity - icons:
error=> 🚨,warning=>⚠️ ,info=> ℹ️ (or nothing?) - message (nice-to-have: transform MD to HTML)
- source file path - file links part of Git URLs in Markdown reports and diffs #149
- source file line(s) -
7if only start line,7-10if both (different) start and end lines,-if no lines
- severity - icons:
- audit description (if present)
- link to audit docs (if present)
- audit detail:
- heading 2: "About"
- Report was created by Code PushUp on DATE.
- as URL, use CLI package README link for now (later: our website)
- date formatted as
new Date().toString()
- table with metadata about the run:
- commit message and truncated hash (current commit in Git) - commit links part of Git URLs in Markdown reports and diffs #149
- package version
- duration (formatted in seconds)
- number of plugins
- number of audits (in total)
- number of categories
- The following plugins were run:
- table with metadata for each plugin:
- plugin title, with link to plugin docs (if present)
- number of audits (from this plugin)
- plugin's package version (if present, otherwise
n/a) - plugin's duration
Example
Follows below line...
Code PushUp report
| 🏷️ Category | ⭐ Score | 🛡️ Audits |
|---|---|---|
| Performance | 🟢 92 | 5 |
| Code style | 🟡 54 | 12 |
🏷️ Categories
Performance
🟢 Score: 92
- 🟢 Performance (Lighthouse):
- 🟨 Largest Contentful Paint - 1.5 s
- 🟥 Disallow missing
keyprops in iterators/collection literals (ESLint) - 1 error
Code style
🟡 Score: 54
- 🟥 Require or disallow method and property shorthand syntax for object literals (ESLint) - 4 warnings
- 🟩 Require
letorconstinstead ofvar(ESLint) - 0
🛡️ Audits
Largest Contentful Paint (Lighthouse)
🟨 1.5 s (score: 75)
Disallow missing key props in iterators/collection literals (ESLint)
🟥 1 error (score: 0)
Issues
| Severity | Message | Source file | Line(s) |
|---|---|---|---|
| 🚨 error | Missing "key" prop for element in iterator | src/components/TodoList.jsx |
7 |
ESLint rule jsx-key, from react plugin. 📖 Docs
Require or disallow method and property shorthand syntax for object literals (ESLint)
🟥 4 warnings (score: 0)
Issues
| Severity | Message | Source file | Line(s) |
|---|---|---|---|
| Expected property shorthand | src/hooks/useTodos.js |
19 | |
| Expected property shorthand | src/hooks/useTodos.js |
32 | |
| Expected property shorthand | src/hooks/useTodos.js |
33 |
ESLint rule object-shorthand. 📖 Docs
Require let or const instead of var (ESLint)
🟩 0 (score: 100)
ESLint rule no-var. 📖 Docs
About
Report was created by Code PushUp on Tue Oct 24 2023 09:38:51 GMT+0200 (Central European Summer Time).
| Commit | Version | Duration | Plugins | Categories | Audits |
|---|---|---|---|---|---|
| Implement todos list (3ac01d1) | 0.1.0 |
1.6 s | 2 | 3 | 25 |
The following plugins were run:
| Plugin | Audits | Version | Duration |
|---|---|---|---|
| ESLint | 20 | 0.1.0 |
0.3 s |
| Lighthouse | 5 | 0.1.0 |
1.2 s |
Activity
BioPhoton commentedon Oct 25, 2023
I recommend listing also the audits weighted with 0 as they might be critical things but not possible to score...
Thoughts?
BioPhoton commentedon Oct 25, 2023
If we would inline the score of a category we could save some lines
BEFORE:
Performance
🟢 Score: 92
AFTER:
🟢 Performance: 92
BioPhoton commentedon Oct 25, 2023
Remark on the "Categories Section".
Let's also show how multiple resulting issues in the examples e.g. 1 error, 4 warnings
The order should be:
error,warning,info.Same for the issue table.
BEFORE:
Code style
🟡 Score: 54
AFTER:
Code style
🟡 Score: 54
BioPhoton commentedon Oct 25, 2023
It is a bit "scrolly" to focus on a goal e.g. performance and have to scroll from the list of categories down to the audits and look up where it. Then start again in categories.
In stead of grouping by data attribute we could take the view of our personas (we have to write them up) and try to make it easy for them to fulfil their needs. Getting better numbers in goal XYZ.
I would create a persona, and put an issue in the back lock. This can be considered when we have the PR comment version
BioPhoton commentedon Oct 25, 2023
I believe that the colors ad quite some complexity. From a product owner perspective I would try to evaluate the need for that feature with the customer.
If we keep it we have to consider that not all goals are "green" at a score of 90 but also can be e.g. 60.
I guess we can understand that problem better when we have implemented the assertion logic and the user defined budgets
matejchalk commentedon Oct 26, 2023
Hmm, there's quite a simple way to solve that, I think 🤔 In each audit's detail, we could include something like "Referenced by categories: Performance, Code style.", that way you can easily jump back to the category.
I don't want to change the scope of this issue, but I can create a follow-up issue for that. What do you think?
Sure, personas need to be considered. Again, let's separate that discussion from this issue. But I can mention quickly how I think about it conceptually in relation to how the report is structured:
The color is always gonna be a bit of a simplification, because there's a limited set of values ("good", "bad" and "something in between"). But I don't think that's a problem if we also show the actual score next to it, the goal is just to add some extra visual cues. It's possible the thresholds won't be universal and we can solve that by enabling the color grading to be configurable, but I wouldn't complicate it too early until we get some more definitive customer feedback.
matejchalk commentedon Oct 27, 2023
Sorry, didn't scroll up all the way and I missed some of the earlier comments 🤦
Yes, I didn't mention this explicitly (added it now), but I would simply list every audit/group the category references, regardless of weight.
That also looks good for categories, in my opinion. But I think we should be consistent with audits as well (a result should ideally always look the same, e.g. in my proposal it's always
<color> <text with main value in bold>), and there the title would become too cluttered. And I wouldn't be worried about saving lines here, the main thing is readability.Yes, that's how it will be formatted 🙂 It's actually already handled by the ESLint plugin (implementation here - you can see it uses helper functions from
@code-pushup/utils) and provided as thedisplayValue.This actually makes the formatter's logic quite simple, it basically just has to do
chalk.bold(audit.displayValue || audit.value.toString()).BioPhoton commentedon Oct 28, 2023
Yes let's create something and use it a bit to feel it in real life.
test(utils): correct report-to-md unit test
feat(utils): implement report.md formatting (#196)