Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve build/generate-sri.js regex #29753

Merged
merged 10 commits into from Nov 21, 2020
Merged

Conversation

Noah2610
Copy link
Contributor

@Noah2610 Noah2610 commented Nov 27, 2019

When using single-quotes in config.yml, the previous
regular expression in build/generate-cli.js wasn't working correctly,
it was replacing ALL string values with hashes.
Now both double- and single-quotes can be used in config.yml,
and the RegExp will work as expected.


I have tested the script with both ' and " used in config.yml, and it seems to work properly!
The changes to config.yml haven't been committed, though.

Fixes #29741

@Noah2610 Noah2610 requested a review from a team as a code owner November 27, 2019 13:10
@Noah2610
Copy link
Contributor Author

Sorry, my bad! I forgot to run npm run test! Thanks for fixing my mistake...

@Noah2610 Noah2610 requested a review from a team as a code owner November 27, 2019 13:16
@XhmikosR
Copy link
Member

@Noah2610 please remove any dist files :)

@Noah2610
Copy link
Contributor Author

@XhmikosR Remove the dist files from my fork? Or where do you mean?

@XhmikosR
Copy link
Member

From your branch. No dist files should be touched.

@XhmikosR
Copy link
Member

Do not delete the dist files... just don't touch them in your patch

When using single-quotes in config.yml, the previous
regular expression in build/generate-cli.js wasn't working correctly,
it was replacing ALL string values with hashes.
Now both double- and single-quotes can be used in config.yml,
and the RegExp will work as expected.
@Noah2610
Copy link
Contributor Author

I'm sorry, I'm not following... I haven't touched the dist/ files in my original commit. I'm sorry for any trouble I'm causing! Would you mind explaining to me what I'm doing wrong?

@XhmikosR
Copy link
Member

You deleted the dist files when you shouldn't have touched them in the first place. I fixed it, please don't overwrite the changes.

@mdo
Copy link
Member

mdo commented Jun 16, 2020

Up to you whether we keep this @XhmikosR.

@mdo mdo changed the base branch from master to main June 16, 2020 20:13
@XhmikosR
Copy link
Member

Still need to test this out, I forgot it even existed :P

@XhmikosR XhmikosR self-requested a review June 18, 2020 10:03
@mdo
Copy link
Member

mdo commented Sep 18, 2020

I mean I'm fine leaving it as-is given it works for us. Still your call.

@XhmikosR
Copy link
Member

@Noah2610 can you please make a https://regex101.com/ example? We don't really hit this issue, but if it helps other people we can consider merging it.

PS. any reason you don't use non-capturing groups?

@Noah2610
Copy link
Contributor Author

@XhmikosR Here's the regex101: https://regex101.com/r/Yhl9vL/1
Note, that the \w+_hash part in the regex is only for regex101, in the generate-sri.js script it is replaced with the property name.
Also note, that some values in the regex101 string example use " or ' quotes, to showcase both cases.

I don't remember why I went so heavy on the capturing-groups. I cleaned it up a bit in the last commit.

@XhmikosR
Copy link
Member

Perfect, thanks!

I can quickly confirm this is fine since the Netlify preview doesn't throw any SRI mismatch errors, but I'll test it later more and merge it soon. I'll also backport this to v4-dev if/after this lands.

@XhmikosR
Copy link
Member

One last thing: any reason you switched to * from +? I'm not sure this change is needed.

@Noah2610
Copy link
Contributor Author

Did I? I don't see any + in the regex in the old commit.

@XhmikosR
Copy link
Member

XhmikosR commented Nov 21, 2020

s+, \S+.

We might as well specify m and use $ since you added ^.

EDIT: scratch the comment about m, forgot for a moment we are using sed there :)

@XhmikosR
Copy link
Member

Perfect, I think everything's is fine, but I'll test later again.

Thanks!

@Noah2610
Copy link
Contributor Author

Noah2610 commented Nov 21, 2020

https://regex101.com/r/Yhl9vL/3

I added a badly formatted yaml line to the end of the example string to showcase why \s* may be better than \s+.
But it's your call, should work for basically all cases anyway.

EDIT: Although the whitespace at the start of the line is invalid yaml in this case, so the first \+ should definitely stay.

@XhmikosR
Copy link
Member

So, basically, only the second \s+ is valid, right? If so, feel free to push the change and update the regex101 link one last time :)

@Noah2610
Copy link
Contributor Author

The first \s+ is valid (to match with whitespace at the start of the line),
The second \s+ should be * to match the case where there's no whitespace between the : and the value.
And the third \S+ (non-whitespace) should probably also be * to allow for empty hash values by default, for example if the user deletes all hashes (empties the strings) before they run the script.

Another thing I noticed, is that we probably shouldn't have a $ at the end, to allow for trailing whitespace or comments at the end of the line.

Here's the updated regex101: https://regex101.com/r/Yhl9vL/4

I'll update the script and test it locally before I commit.

@XhmikosR
Copy link
Member

Well, TBH, I don't think we should cover all those cases. The script is supposed to be used internally, and it's OK to make a small change to take care of your issue, but I don't think we should care about all the other cases.

@Noah2610
Copy link
Contributor Author

I understand.
I have the updated regex, let me know if you want me to commit, or if we should just leave it as is.

@Noah2610
Copy link
Contributor Author

Also, nevermind, yaml requires a space after the :, I was wrong about that being valid yaml.
https://stackoverflow.com/questions/42124227/why-does-the-yaml-spec-mandate-a-space-after-the-colon
https://regex101.com/r/Yhl9vL/5

@XhmikosR
Copy link
Member

OK, so what's your final suggestion?

@Noah2610
Copy link
Contributor Author

I'd use the regex in the last version of the regex101: https://regex101.com/r/Yhl9vL/5
It just changes the \S+ to \S* and removes the $.
Doesn't break anything and it isn't a big regex change. I can commit it right away.

@XhmikosR
Copy link
Member

Cool, let's go with that then 👍

@XhmikosR XhmikosR added this to Inbox in v5.0.0-beta1 via automation Nov 21, 2020
@XhmikosR XhmikosR added this to Inbox in v4.6.0 via automation Nov 21, 2020
@XhmikosR XhmikosR removed request for a team November 21, 2020 09:38
@XhmikosR XhmikosR moved this from Inbox to Review in v5.0.0-beta1 Nov 21, 2020
@XhmikosR XhmikosR changed the title Fix build/generate-sri.js regex (#29741) Improve build/generate-sri.js regex Nov 21, 2020
v5.0.0-beta1 automation moved this from Review to Approved Nov 21, 2020
@XhmikosR XhmikosR merged commit 21737ed into twbs:main Nov 21, 2020
v5.0.0-beta1 automation moved this from Approved to Done Nov 21, 2020
@XhmikosR XhmikosR moved this from Inbox to Cherry-picked/Manually backported in v4.6.0 Nov 21, 2020
XhmikosR added a commit that referenced this pull request Nov 21, 2020
When using single-quotes in config.yml, the previous
regular expression in build/generate-cli.js wasn't working correctly,
it was replacing ALL string values with hashes.
Now both double- and single-quotes can be used in config.yml,
and the RegExp will work as expected.

Also, allow trailing whitespaces and empty ("") values to be matched.

Co-authored-by: XhmikosR <xhmikosr@gmail.com>
XhmikosR added a commit that referenced this pull request Nov 23, 2020
When using single-quotes in config.yml, the previous
regular expression in build/generate-cli.js wasn't working correctly,
it was replacing ALL string values with hashes.
Now both double- and single-quotes can be used in config.yml,
and the RegExp will work as expected.

Also, allow trailing whitespaces and empty ("") values to be matched.

Co-authored-by: XhmikosR <xhmikosr@gmail.com>
@XhmikosR XhmikosR moved this from Cherry-picked/Manually backported to Shipped in v4.6.0 Nov 24, 2020
XhmikosR added a commit that referenced this pull request Nov 24, 2020
When using single-quotes in config.yml, the previous
regular expression in build/generate-cli.js wasn't working correctly,
it was replacing ALL string values with hashes.
Now both double- and single-quotes can be used in config.yml,
and the RegExp will work as expected.

Also, allow trailing whitespaces and empty ("") values to be matched.

Co-authored-by: XhmikosR <xhmikosr@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
v4.6.0
Shipped
v5.0.0-beta1
  
Done
Development

Successfully merging this pull request may close these issues.

build/generate-sri.js doesn't work if single-quotes are used in config.yml
3 participants