Skip to content

Commit

Permalink
feat(gatsby): lazy bundle page components in dev server (#27884)
Browse files Browse the repository at this point in the history 
* feat(gatsby): lazy compile page components in dev server

This means we only compile the core runtime code initially — page components are only compiled when the user visits the page.

* Fix lint

* Cleanup logs

* cleanups

* fix tests

* Don't bother with navigating — if webpack doesn't update — the user probably has errors in their code or something bigger is wrong so let them just refresh, etc.

* Update packages/gatsby/src/bootstrap/requires-writer.ts

Co-authored-by: Ward Peeters <ward@coding-tech.com>

* Update packages/gatsby/cache-dir/dev-loader.js

Co-authored-by: Ward Peeters <ward@coding-tech.com>

* Fix lint error

* Wait until we know staticQueryHashes are written to page-data.json files

We only discover static queries for pages after the component is compiled.

Which means when we lazily add page components, we initially don't know
what static queries should be loaded. To track this, we mark page-data.json
files as not bundled & only after the staticQueryHashes is written do we
accept the page as loaded.

* Disable check in production

* Only trigger compilation/page writing once

* Also pull xhr loaded static queries for <StaticQuery>

* instance isn't defined in builds apparently

* Work around webpack hot reloading bug

* For static queries, data pushed from websocket is the most fresh

* Centralize logic and prevent it from repeatedly hitting the backend

* fix ordering of merge

* Comment and add timeout to ensure page-data writes happen after the webpack compilation

* Only add notInDevBundle during development

* Add back mistakenly removed code

* Didn't need this

* Implement @pieh's idea for telling runtime that dev bundle is loaded

* Fix tests

* Remove logs

* Put changes behind GATSBY_EXPERIMENT_LAZY_DEVJS flag

* Add temporary fork of development-runtime tests for lazy devjs PR

* Remove duplicated tests w/ cleaner method @pieh suggested

* Address @pieh feedback

* Update packages/gatsby/src/utils/start-server.ts

Co-authored-by: Michal Piechowiak <misiek.piechowiak@gmail.com>

* Remove global middleware

* This isn't true

* fix lint

Co-authored-by: Ward Peeters <ward@coding-tech.com>
Co-authored-by: Michal Piechowiak <misiek.piechowiak@gmail.com>
  • Loading branch information
@KyleAMathews @wardpeet @pieh
3 people committed Nov 17, 2020
1 parent 179694a commit 04349a0
Show file tree
Hide file tree
Showing 18 changed files with 942 additions and 15 deletions.
26 changes: 26 additions & 0 deletions .circleci/config.yml
View file
Expand Up @@ -117,6 +117,23 @@ aliases:
- notify-status:
condition: << parameters.nightly >>

e2e_tests_development_runtime_lazy_alias:
&e2e_tests_development_runtime_lazy_alias
<<: *e2e-executor
parameters:
nightly:
type: boolean
default: false
environment:
CYPRESS_PROJECT_ID: ihj5mz
CYPRESS_RECORD_KEY: 01acdce8-75bd-4280-9839-6cb215b2c84b
steps:
- e2e-test:
test_path: e2e-tests/lazy-development-runtime
skip_file_change_test: << parameters.nightly >>
- notify-status:
condition: << parameters.nightly >>

e2e_tests_gatsby-image_alias: &e2e_tests_gatsby-image_alias
<<: *e2e-executor
parameters:
Expand Down Expand Up @@ -332,6 +349,13 @@ jobs:
e2e_tests_development_runtime:
<<: *e2e_tests_development_runtime_alias

e2e_tests_development_runtime_lazy_devjs:
<<: *e2e_tests_development_runtime_alias
environment:
GATSBY_EXPERIMENT_LAZY_DEVJS: true
CYPRESS_PROJECT_ID: ihj5mz
CYPRESS_RECORD_KEY: 01acdce8-75bd-4280-9839-6cb215b2c84b

e2e_tests_development_runtime_with_experimental_react:
<<: *e2e_tests_development_runtime_alias

Expand Down Expand Up @@ -605,6 +629,8 @@ workflows:
<<: *e2e-test-workflow
- e2e_tests_development_runtime:
<<: *e2e-test-workflow
- e2e_tests_development_runtime_lazy_devjs:
<<: *e2e-test-workflow
- e2e_tests_production_runtime:
<<: *e2e-test-workflow
- themes_e2e_tests_production_runtime:
Expand Down
20 changes: 20 additions & 0 deletions packages/gatsby/cache-dir/__tests__/__snapshots__/dev-loader-lazy.js.snap
View file
@@ -0,0 +1,20 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`Dev loader loadPage should be successful when component can be loaded 1`] = `
Object {
"component": Object {
"chunkName": "bar",
},
"json": Object {
"pageContext": "something something",
},
"page": Object {
"componentChunkName": "chunk",
"matchPath": undefined,
"path": "/mypage/",
"staticQueryHashes": Array [],
"webpackCompilationHash": "123",
},
"staticQueryResults": Object {},
}
`;

1 comment on commit 04349a0

@Anmol368
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (process.env.GATSBY_EXPERIMENT_LAZY_DEVJS) {
const bodyParser = require(body-parser)
const { boundActionCreators } = require(../redux/actions)
const { createClientVisitedPage } = boundActionCreators
// Listen for the client marking a page as visited (meaning we need to
// compile its page component.
const chunkCalls = new Set()
app.post(/___client-page-visited, bodyParser.json(), (req, res, next) => {
if (req.body?.chunkName) {
// Ignore all but the first POST.
if (!chunkCalls.has(req.body.chunkName)) {
// Tell Gatsby there's a new page component to trigger it
// being added to the bundle.
createClientVisitedPage(req.body.chunkName)

      // Tell Gatsby to rewrite the page data for the pages
      // owned by this component to update it to say that
      // its page component is now part of the dev bundle.
      // The pages will be rewritten after the webpack compilation
      // finishes.
      //
      // Set a timeout to ensure the webpack compile of the new page
      // component triggered above has time to go through.
      setTimeout(() => {
        // Find the component page for this componentChunkName.
        const pages = store.getState().pages
        function getByChunkName(map, searchValue): void | string {
          for (const [key, value] of map.entries()) {
            if (value.componentChunkName === searchValue) return key
          }

          return undefined
        }
        const pageKey = getByChunkName(pages, req.body.chunkName)

        if (pageKey) {
          const page = pages.get(pageKey)
          if (page) {
            store.dispatch({
              type: `ADD_PENDING_TEMPLATE_DATA_WRITE`,
              payload: {
                pages: [
                  {
                    componentPath: page.component,
                  },
                ],
              },
            })
          }
        }
        chunkCalls.add(req.body.chunkName)
      }, 20)
    }
    res.send(`ok`)
  } else {
    next()
  }
})

}

/**

  • Set up the express app.
    **/

Please sign in to comment.