Add Azure Pipelines CI #2299

damccorm · 2019-03-08T16:04:32Z

The Change

This PR adds Azure Pipelines to perform CI builds and PR builds. Pipelines offers several key advantages:

Single definition for all platforms (Linux/Windows/Mac) reduces the amount of time spent maintaining CI.
Faster overall build times because of free parallelism - Pipelines offers 10 free concurrent jobs and unlimited build minutes for all OSS projects. Plus, for big projects like this one, Microsoft is happy to raise the concurrency limit, so we could have 15 or 20 parallel jobs. This should cut max build times to 4 or 4.5 minutes on Pipelines (it seems to vary a lot for Appveyor, but I've seen anything from 6 - 10 minutes in previous builds).
Free MacOS builds - right now we're not building on MacOS at all, but some issues appear only on that OS.

Disclaimer

Full disclosure, I'm an engineer on the Pipelines team, but I also think Pipelines provides some pretty significant advantages here.

Additional Info

You can see that this works in my pipeline.

With test reporting!

phated · 2019-03-08T18:07:58Z

This is actually something I've been considering.

I won't have any time today to dig in but I'll try to find time soon.

Can you remove the package-lock file that was generated? I haven't yet added the npmrc file that stops them from being generated.

damccorm · 2019-03-08T18:13:10Z

Whoops, didn't realized I'd added that, taken care of 😄

phated

@damccorm thanks for submitting this! I'm really excited about getting azure pipelines set up. I have a few comments/questions.

phated · 2019-03-10T20:12:26Z

.azure-pipelines-steps.yml

+
+- task: PublishTestResults@2
+  inputs:
+    testResultsFiles: '**/xunit.xml'


So azure pipelines only speaks xunit?

With this input it does, but it can also use JUnit, NUnit, VSTest, xUnit, or cTest, we'd just need to change testResultsFiles to something like **/junit.xml. I believe if we want we could also do something like **/TEST-**.xml, this input really just works as a pattern matcher I think.

I used xunit since that matches the custom reporter I added (more on that below).

@damccorm It looks like mocha (even the version we use) ships with an XUnit reporter. Can we just use that? We can have an azure script that does mocha --async-only --reporter xunit

That way we can drop the extra reporter.

That works, the only downside is we don't get great output in the console.

What we get with multi-reporter:

What we get with xunit reporter:

I have this change in PR, if you're comfortable with that tradeoff I'll merge just let me know.

Oh actually, on closer inspection, this doesn't get written to a file which is necessary for us to report results. We can pipe it out to a file, but that would mean we would get no output at all on the console.

We can use the -O output=test.xunit flag to specify the output file. I was thinking we could just run the test suites as separate phases, one with the spec reporter and one with the xunit reporter.

Makes sense to me! Updated accordingly

.azure-pipelines-steps.yml

.azure-pipelines.yml

package.json

phated · 2019-03-12T20:19:59Z

@damccorm Just a few more things I followed up on. Also, do you have any guidance on how I signup and get this configured to work under a gulpjs organization for Azure Pipelines?

phated · 2019-03-12T20:21:01Z

Oh, also - does Azure Pipelines support a .ci/ directory? I'd like to start moving things there if possible.

damccorm · 2019-03-13T14:25:36Z

Oh, also - does Azure Pipelines support a .ci/ directory? I'd like to start moving things there if possible.

Sure, I moved it in there in my PR.

do you have any guidance on how I signup and get this configured to work under a gulpjs organization for Azure Pipelines?

The easiest way to get this done is probably to install the GitHub app after you've merged in this PR. That will walk you through the process of creating an organization and project. From there you can follow the getting started docs here.

One thing to call out is that since we moved the yaml to the .ci folder it won't be automatically discovered immediately when you create a build pipeline. That just means you'll want to select the Existing Azure Pipelines YAML file configuration and set /.ci/.azure-pipelines.yml as your path.

Let me know if anything is unclear as well, happy to help however I can!

damccorm · 2019-03-13T14:29:56Z

One more thing - I noticed that both the organization names gulp and gulpjs are taken. I'm happy to transfer gulpci to you as an org name if you can't find a better one. It also looks like gulpjs is currently owned by team@gulpjs.com, so maybe your team already has one?

Regardless, installing the GitHub app is still the best way to get going.

phated · 2019-03-13T21:33:47Z

@damccorm yeah, I did set one up a long time ago but I gave up when I ran into tons of XML. Is the YAML stuff new?

~~Can I use the existing team if I install it as a GitHub App?~~ I just connected it and it looks like it works fine.

Get rid of extra devDep

damccorm · 2019-03-13T21:57:41Z

Is the YAML stuff new?

Yeah, it was added a little more than half a year ago as the default experience. It was part of a larger push (which is still ongoing) to offer a better OSS experience in particular.

Can I use the existing team if I install it as a GitHub App?

Yep! It should give you an opportunity to sign into an existing organization.

phated · 2019-03-20T19:06:30Z

Sorry for the delay. This is looking great!

phated · 2019-03-20T19:26:06Z

@damccorm I'm looking at the docs for secure variables and it seems to indicate there should be a 🔒 icon in the variables panel but I don't have that. How do I secure this coveralls token? Also, by using the coveralls token as a secure token, it says it won't be available to PRs - won't that break our coveralls Status API integration for PRs?

damccorm · 2019-03-20T19:57:40Z

there should be a 🔒 icon in the variables panel but I don't have that

Are you looking at this page or somewhere else? This is what I see (with the secure checkbox column highlighted):

If not could you provide a little more info on what you're looking at? To get to that page, you can follow these steps:

Edit pipeline

Choose variables from the dropdown here:

Also, by using the coveralls token as a secure token, it says it won't be available to PRs - won't that break our coveralls Status API integration for PRs?

I missed that you were doing this. The answer is yes by default, but you can configure this behavior from the same menu (under the triggers tab)

If you do this, I would be careful though. The reason that secrets aren't enabled on forked builds is that someone could change a script that a secret is mapped into and do something malicious (e.g. send themselves the token). They can't change the yaml that gets executed though (or at least it won't get executed until merged), so a change you could make to ensure this doesn't happen is instead of calling npm run coveralls running the actual script (npm run cover && istanbul-coveralls). I'll add a PR to do this.

phated · 2019-03-20T21:12:08Z

@damccorm thanks for following up. I have totally different variables view than you. Is it possible that I'm on a newer/older version of the UI because I signed up so long ago? I also noticed that other stuff in the documentation didn't match my UI.

phated · 2019-03-20T21:14:56Z

@damccorm this is what I'm seeing for my "variables" screen

damccorm · 2019-03-20T22:16:42Z

Huh, that's weird, I haven't seen that view before (I joined Pipelines relatively recently though, so there's a chance it predates me). One thing that you could do is check to make sure the new UI didn't somehow get disabled in the preview features. For me you can access this here, hopefully this works for you as well:

damccorm · 2019-03-20T22:17:31Z

I'll also reach out internally and get back to you on this one, I don't know exactly why its behaving this way

phated · 2019-03-20T23:13:47Z

@damccorm there was some stuff disabled in the "Preview Features" but I've now enabled everything and still don't have that UI. If it gives any indicators, the URL I'm accessing is https://dev.azure.com/gulpjs/gulp/_apps/hub/ms.vss-ciworkflow.build-ci-hub?_a=edit-build-definition&id=1&view=Tab_Variables

phated · 2019-03-21T22:37:49Z

@damccorm LMAO - that UI comes from having the sidebar opened. I just closed it and noticed more to the UI. I never even tried to scroll that pane because there was no indicator that it had overflowed the panel. Any chance you could add that to your issue tracker????

damccorm · 2019-03-22T03:14:07Z

Yikes. Yeah, that's not good, we'll address that.

Danny McCormick added 2 commits March 7, 2019 17:33

Add azure pipelines

0c240e0

Reorganize

e0852ef

stephenlacy approved these changes Mar 8, 2019

View reviewed changes

Delete package-lock.json

e283658

phated reviewed Mar 10, 2019

View reviewed changes

Use more versions on mac

0e9394a

Danny McCormick added 3 commits March 13, 2019 09:43

Move pipelines into ci folder

8c96cb9

Merge branch 'master' of https://github.com/damccorm/gulp

5883b7f

Get rid of extra devDep

73b0ba4

Danny McCormick added 5 commits March 13, 2019 17:45

Add seperate step for xml

6538d76

Add seperate step for xml

ee50b1b

Find test files

ebf0aaa

Fix file finding

bfe888b

Get rid of extra devDep

12c6490

Get rid of extra devDep

phated merged commit 2e46ccd into gulpjs:master Mar 20, 2019

damccorm mentioned this pull request Mar 20, 2019

Explicitly run coveralls step #2306

Closed

phated pushed a commit that referenced this pull request Mar 24, 2019

Build: Add Azure Pipelines CI (#2299)

b2c6c7e

damccorm mentioned this pull request Mar 25, 2019

Consider adding Azure Pipelines badge #2310

Merged

snyk-bot mentioned this pull request Oct 26, 2019

[Snyk] Upgrade gulp from 4.0.0 to 4.0.2 ajesse11x/PowerShell#3

Open

snyk-bot mentioned this pull request Nov 14, 2019

[Snyk] Upgrade gulp from 4.0.0 to 4.0.2 Matthelonianxl/flight-manual.atom.io#2

Open

snyk-bot mentioned this pull request Feb 22, 2020

[Snyk] Upgrade gulp from 4.0.0 to 4.0.2 raindigi/cloudcannon-suite#5

Open

snyk-bot mentioned this pull request Mar 17, 2020

[Snyk] Upgrade gulp from 4.0.0 to 4.0.2 cdcc1969/etherwallet#7

Open

snyk-bot mentioned this pull request Apr 1, 2020

[Snyk] Upgrade gulp from 4.0.0 to 4.0.2 baophucct/PowerShell#3

Open

NOUIY mentioned this pull request May 18, 2024

[Snyk] Upgrade gulp from 4.0.0 to 4.0.2 NOUIY/flight-manual.atom.io#601

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Azure Pipelines CI #2299

Add Azure Pipelines CI #2299

damccorm commented Mar 8, 2019

phated commented Mar 8, 2019

damccorm commented Mar 8, 2019

phated left a comment

phated Mar 10, 2019

damccorm Mar 11, 2019

phated Mar 12, 2019

damccorm Mar 13, 2019

damccorm Mar 13, 2019

phated Mar 13, 2019

damccorm Mar 13, 2019

phated commented Mar 12, 2019

phated commented Mar 12, 2019

damccorm commented Mar 13, 2019

damccorm commented Mar 13, 2019

phated commented Mar 13, 2019 •

edited

damccorm commented Mar 13, 2019

phated commented Mar 20, 2019

phated commented Mar 20, 2019

damccorm commented Mar 20, 2019

phated commented Mar 20, 2019

phated commented Mar 20, 2019

damccorm commented Mar 20, 2019

damccorm commented Mar 20, 2019

phated commented Mar 20, 2019

phated commented Mar 21, 2019 •

edited

damccorm commented Mar 22, 2019

Add Azure Pipelines CI #2299

Add Azure Pipelines CI #2299

Conversation

damccorm commented Mar 8, 2019

phated commented Mar 8, 2019

damccorm commented Mar 8, 2019

phated left a comment

Choose a reason for hiding this comment

phated Mar 10, 2019

Choose a reason for hiding this comment

damccorm Mar 11, 2019

Choose a reason for hiding this comment

phated Mar 12, 2019

Choose a reason for hiding this comment

damccorm Mar 13, 2019

Choose a reason for hiding this comment

damccorm Mar 13, 2019

Choose a reason for hiding this comment

phated Mar 13, 2019

Choose a reason for hiding this comment

damccorm Mar 13, 2019

Choose a reason for hiding this comment

phated commented Mar 12, 2019

phated commented Mar 12, 2019

damccorm commented Mar 13, 2019

damccorm commented Mar 13, 2019

phated commented Mar 13, 2019 • edited

damccorm commented Mar 13, 2019

phated commented Mar 20, 2019

phated commented Mar 20, 2019

damccorm commented Mar 20, 2019

phated commented Mar 20, 2019

phated commented Mar 20, 2019

damccorm commented Mar 20, 2019

damccorm commented Mar 20, 2019

phated commented Mar 20, 2019

phated commented Mar 21, 2019 • edited

damccorm commented Mar 22, 2019

phated commented Mar 13, 2019 •

edited

phated commented Mar 21, 2019 •

edited