feat(server): support socket path in server config

[MaximeHeckel]

Fixes #385

• Add sqlSocketPath option support in the config
• Validate config UX with project maintainers
• Add tests for new socket path option in config (Looking for guidance here 😄 )
• Mention sqlSocketPath in docs/configuration.md

As mentioned in the ticket, my team and I were looking to set up lighthouse-ci server on Google Cloud Run (serverless containers) with a Google Cloud SQL database (Postgres) and our only possibility to link the two was to use a socket path instead of a connection URL.

Let me know what you think of this config, there's probably a few things that still need to be done or validated in terms of how you want the config to look like.
Also @patrickhulce mentioned that testing might be quite hard for this one, so I'm looking for any proposal on how I could tackle this 😄

[googlebot]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[MaximeHeckel]

@googlebot I signed it!

[googlebot]

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

[patrickhulce]

impl looks good to me! want to create another copy of mysql or postgres server test suite that uses an environment variable for the socket path to use?

it would look pretty similar to https://github.com/GoogleChrome/lighthouse-ci/blob/master/packages/server/test/mysql-server.test.js just with your new sqlSocketPath option

[MaximeHeckel]

@patrickhulce thank you for the suggestions! Will address the comments and add tests

[MaximeHeckel]

@patrickhulce maybe you have a bit more context regarding how to use the mysql service with Github Actions but I currently don't seem to have permissions to connect through socket path in CI / Linux:
SequelizeConnectionError: connect EACCES /var/lib/mysql/mysql.sock

[patrickhulce]

maybe you have a bit more context regarding how to use the mysql service with Github Actions but I currently don't seem to have permissions to connect through socket path in CI / Linux

Well we'll need to start another copy of mysql that exposes the socket path, right now it's just running in a docker container on port 33306, so that socket file won't exist.

Looks like it's not too bad though https://superuser.com/questions/1411402/how-to-expose-linux-socket-file-from-docker-container-mysql-mariadb-etc-to

So steps...

1. We create a copy of the mysql entry currently under services but have it use a socket instead.
2. Expose the socket folder to the host image like the answer linked above
3. Do the same steps for creating our test database like we do for the port-based on but using the host socket file.
4. Set the environment variable like you do now to get the tests to use this socket file.

[patrickhulce]

fyi @MaximeHeckel you might want to create subdirs so the docker mount doesn't try to clobber an existing directory with the mount I don't think setting /mnt for both postgres and mysql will work

[patrickhulce]

sorry this is such a pain to debug 😅

[MaximeHeckel]

you might want to create subdirs so the docker mount doesn't try to clobber an existing directory

Yeah ran into that issue originally with /tmp

sorry this is such a pain to debug 😅

and haha no worries I'm used to use gh action and spending hours of trial and error until a workflow works

[MaximeHeckel]

Is there a way wit gh actions to create the directories before the service starts? I believe the directories of the host need to be present before the service start right?

[patrickhulce]

Hm I'm not sure to be honest. I would assume there should be but I haven't tried before

[MaximeHeckel]

Did a bit of investigation yesterday night, it doesn't look like it's possible sadly. (Couldn't find a lot of documentations around volumes, tried to work around it but didn't find any satisfying solution to this)
What we can do maybe is use another path than /mnt? Do you have any suggestions? (I tried /tmp as well but yarn wouldn't be able to successfully run after putting the sockets there)

[patrickhulce]

have you tried using the workspace dir yet? something like ${{ github.workspace }}/mysql:/var/run/mysqld? we just need it to be predictable and accessible

[MaximeHeckel]

Just gave it a try, but like in the previous attempt the mysqlsocket service doesn't seem to be able to start: https://github.com/GoogleChrome/lighthouse-ci/pull/389/checks?check_run_id=919336503

[patrickhulce]

Oof, yeah this is really rough @MaximeHeckel I'm about to give up as well ☹️ https://github.com/GoogleChrome/lighthouse-ci/actions

I'm inclined to just skip the test here at this point since there's not explicit configuration involved if things are working on your end. sockets through docker just seem rough :/

leave the test file in play at least and maybe I'll play around with github actions some more on the subject later. just remove the environment variable for now in the github actions config I guess

[patrickhulce]

@googlebot I consent.

[MaximeHeckel]

I'm inclined to just skip the test here at this point since there's not explicit configuration involved if things are working on your end. sockets through docker just seem rough :/

yeah not an easy one (funny thing is that I used to work for Docker, and I remember struggling as much back then, seems that things haven't improved)

just remove the environment variable for now in the github actions config I guess

Will do that right now

The tests were passing and locally I didn't have any issues
For reference here's the Github Action to prove it 😄 both Postgres and Mysql working through socket: https://github.com/GoogleChrome/lighthouse-ci/actions/runs/184918838

...
2020-07-27T22:59:11.7719998Z PASS packages/server/test/postgres-socket-path-server.test.js (7.029s)
...
2020-07-27T22:59:17.7798161Z PASS packages/server/test/mysql-socket-path-server.test.js (12.73s)
...

[patrickhulce]

Oh wait you got it working!? That's amazing! Ignore my removal comment if you fixed it I thought we were stuck in failing limbo :)

thank you very much for your effort here! 🎉

happy to do it in a followup though if you're just done with this PR 😆

[MaximeHeckel]

Oh wait you got it working!? That's amazing! Ignore my removal comment if you fixed it I thought we were stuck in failing limbo :)

Oh sorry, I might have caused some confusion here:

• I got it working locally
• I got it working if mounting a volume in /mnt for example (that's where the logs you see in my previous come from, and that's where the run I linked in that same comment points to)

But like we investigated yesterday, we couldn't get it to work by mounting a volume in /mnt/mysql or /mnt/postgres because we can't create the folder before these services start.

As it stands now this PR has the tests for both mysql and postgres socket path but they are just not running because like you suggested I removed the environment variable for their respective socket paths and thus the tests are skipped.

[patrickhulce]

Ah ok gotcha 👍