Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: contentful/contentful-import
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: ee14335db47403cb18121a2c664511e8b1552f32
Choose a base ref
...
head repository: contentful/contentful-import
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: b1a2114dc2b83bf2d5f986b6e16a0fef33332fb7
Choose a head ref

Commits on Mar 17, 2021

  1. refactor: rewrite push-to-space in async/await

    BREAKING CHANGE: Requires Node 12 or higher
    TimBeyer authored and damienxy committed Mar 17, 2021
    Copy the full SHA
    2d9d1f2 View commit details
  2. Copy the full SHA
    1a632c7 View commit details
  3. Copy the full SHA
    ac61587 View commit details
  4. chore: fix linting

    Co-authored-by: Tim Beyer <TimBeyer@users.noreply.github.com>
    damienxy and TimBeyer committed Mar 17, 2021
    Copy the full SHA
    1e1a224 View commit details
  5. feat: remove prepublishing delay

    TimBeyer authored and damienxy committed Mar 17, 2021
    Copy the full SHA
    87dbb4d View commit details
  6. Copy the full SHA
    74e3872 View commit details
  7. Copy the full SHA
    f715c3c View commit details
  8. Copy the full SHA
    7cdec84 View commit details
  9. Copy the full SHA
    127fe28 View commit details
  10. Copy the full SHA
    4e049c6 View commit details
  11. Copy the full SHA
    a208de8 View commit details
  12. Copy the full SHA
    84af3c8 View commit details
  13. Copy the full SHA
    d10edd6 View commit details
  14. Copy the full SHA
    8011244 View commit details
  15. chore: update readme

    damienxy committed Mar 17, 2021
    Copy the full SHA
    c096663 View commit details
  16. Copy the full SHA
    e43aa9f View commit details

Commits on Mar 19, 2021

  1. Copy the full SHA
    00e34a3 View commit details
  2. Copy the full SHA
    7e5282c View commit details
  3. Copy the full SHA
    172d5c2 View commit details
  4. Copy the full SHA
    a5edd66 View commit details
  5. Merge pull request #416 from contentful/feat/import-exported-asset-files

    Import exported asset files
    damienxy authored Mar 19, 2021
    Copy the full SHA
    4c0fb72 View commit details
  6. Copy the full SHA
    4e1593c View commit details

Commits on Mar 22, 2021

  1. build(deps-dev): bump husky from 5.1.3 to 5.2.0

    Bumps [husky](https://github.com/typicode/husky) from 5.1.3 to 5.2.0.
    - [Release notes](https://github.com/typicode/husky/releases)
    - [Commits](typicode/husky@v5.1.3...v5.2.0)
    
    Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
    dependabot-preview[bot] committed Mar 22, 2021
    Copy the full SHA
    d9575f0 View commit details

Commits on Mar 23, 2021

  1. Copy the full SHA
    ee82389 View commit details
  2. Merge pull request #420 from contentful/EXT-2508

    chore: [EXT-2508] time to use husky v4
    Jwhiles authored Mar 23, 2021
    Copy the full SHA
    bcf98aa View commit details

Commits on Mar 25, 2021

  1. build(deps): bump contentful-management from 7.11.0 to 7.13.0

    Bumps [contentful-management](https://github.com/contentful/contentful-management.js) from 7.11.0 to 7.13.0.
    - [Release notes](https://github.com/contentful/contentful-management.js/releases)
    - [Commits](contentful/contentful-management.js@v7.11.0...v7.13.0)
    
    Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
    dependabot-preview[bot] committed Mar 25, 2021
    Copy the full SHA
    b417af7 View commit details

Commits on Mar 29, 2021

  1. build(deps-dev): bump eslint from 7.22.0 to 7.23.0

    Bumps [eslint](https://github.com/eslint/eslint) from 7.22.0 to 7.23.0.
    - [Release notes](https://github.com/eslint/eslint/releases)
    - [Changelog](https://github.com/eslint/eslint/blob/master/CHANGELOG.md)
    - [Commits](eslint/eslint@v7.22.0...v7.23.0)
    
    Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
    dependabot-preview[bot] committed Mar 29, 2021
    Copy the full SHA
    12db979 View commit details
  2. build(deps): [security] bump y18n from 3.2.1 to 3.2.2

    Bumps [y18n](https://github.com/yargs/y18n) from 3.2.1 to 3.2.2. **This update includes a security fix.**
    - [Release notes](https://github.com/yargs/y18n/releases)
    - [Changelog](https://github.com/yargs/y18n/blob/master/CHANGELOG.md)
    - [Commits](https://github.com/yargs/y18n/commits)
    
    Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
    dependabot-preview[bot] committed Mar 29, 2021
    Copy the full SHA
    7643e54 View commit details

Commits on Mar 30, 2021

  1. build(deps): bump contentful-management from 7.13.0 to 7.13.1

    Bumps [contentful-management](https://github.com/contentful/contentful-management.js) from 7.13.0 to 7.13.1.
    - [Release notes](https://github.com/contentful/contentful-management.js/releases)
    - [Commits](contentful/contentful-management.js@v7.13.0...v7.13.1)
    
    Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
    dependabot-preview[bot] committed Mar 30, 2021
    Copy the full SHA
    89a8b5d View commit details

Commits on Apr 5, 2021

  1. Copy the full SHA
    d67dded View commit details

Commits on Apr 6, 2021

  1. Copy the full SHA
    b6b4098 View commit details
  2. Copy the full SHA
    c6f9ebc View commit details

Commits on Apr 7, 2021

  1. Copy the full SHA
    cb98c22 View commit details
  2. Copy the full SHA
    c5fe53e View commit details

Commits on Apr 8, 2021

  1. Update README.md

    Co-authored-by: Meike P <33579339+damienxy@users.noreply.github.com>
    z0al and damienxy authored Apr 8, 2021
    Copy the full SHA
    9cd0900 View commit details

Commits on Apr 12, 2021

  1. build(deps-dev): bump eslint from 7.23.0 to 7.24.0

    Bumps [eslint](https://github.com/eslint/eslint) from 7.23.0 to 7.24.0.
    - [Release notes](https://github.com/eslint/eslint/releases)
    - [Changelog](https://github.com/eslint/eslint/blob/master/CHANGELOG.md)
    - [Commits](eslint/eslint@v7.23.0...v7.24.0)
    
    Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
    dependabot-preview[bot] committed Apr 12, 2021
    Copy the full SHA
    4ee539f View commit details
  2. Copy the full SHA
    c27f23d View commit details

Commits on Apr 13, 2021

  1. feat: support -H/--header option

    z0al committed Apr 13, 2021
    Copy the full SHA
    c3fbf23 View commit details
  2. Copy the full SHA
    711edca View commit details
  3. test: fix tests

    z0al committed Apr 13, 2021
    Copy the full SHA
    036cb79 View commit details
  4. test: fix tests

    z0al committed Apr 13, 2021
    Copy the full SHA
    f3e7426 View commit details

Commits on Apr 14, 2021

  1. Copy the full SHA
    5e83aaa View commit details
  2. Copy the full SHA
    1c04ff9 View commit details

Commits on Apr 15, 2021

  1. Copy the full SHA
    eaa9073 View commit details
  2. Merge pull request #428 from contentful/custom-headers

    feat: add custom headers support
    z0al authored Apr 15, 2021
    Copy the full SHA
    8d56958 View commit details

Commits on Apr 16, 2021

  1. build(deps): [security] bump ssri from 6.0.1 to 6.0.2

    Bumps [ssri](https://github.com/npm/ssri) from 6.0.1 to 6.0.2. **This update includes a security fix.**
    - [Release notes](https://github.com/npm/ssri/releases)
    - [Changelog](https://github.com/npm/ssri/blob/v6.0.2/CHANGELOG.md)
    - [Commits](npm/ssri@v6.0.1...v6.0.2)
    
    Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
    dependabot-preview[bot] committed Apr 16, 2021
    Copy the full SHA
    e65bf80 View commit details

Commits on Apr 19, 2021

  1. Copy the full SHA
    e5d32c3 View commit details

Commits on Apr 22, 2021

  1. Copy the full SHA
    009f3dd View commit details
  2. Merge pull request #437 from contentful/security/remove-codecov

    security: remove codecov step using bash uploader
    fkiriakos authored Apr 22, 2021
    Copy the full SHA
    9b669ab View commit details
  3. Copy the full SHA
    e7c7ed1 View commit details
Showing with 27,633 additions and 17,860 deletions.
  1. +0 −13 .babelrc
  2. +4 −8 .circleci/config.yml
  3. +0 −13 .dependabot/config.yml
  4. +10 −14 .eslintrc.js
  5. +1 −0 .github/CODEOWNERS
  6. +34 −0 .github/dependabot.yml
  7. +14 −0 .github/workflows/dependabot-approve-and-request-merge.yml
  8. +1 −0 .nvmrc
  9. +38 −7 README.md
  10. +16 −0 babel.config.json
  11. +2 −2 bin/contentful-import
  12. +13 −0 catalog-info.yaml
  13. +32 −30 lib/index.js
  14. +11 −8 lib/parseOptions.js
  15. +84 −59 lib/tasks/get-destination-data.js
  16. +1 −1 lib/tasks/init-client.js
  17. +41 −14 lib/tasks/push-to-space/assets.js
  18. +99 −54 lib/tasks/push-to-space/creation.js
  19. +55 −59 lib/tasks/push-to-space/publishing.js
  20. +195 −154 lib/tasks/push-to-space/push-to-space.js
  21. +1 −1 lib/transform/transform-space.js
  22. +3 −1 lib/transform/transformers.js
  23. +21 −0 lib/usageParams.js
  24. +40 −0 lib/utils/headers.js
  25. +14 −13 lib/utils/schema.js
  26. +1 −1 lib/utils/sort-entries.js
  27. +1 −2 lib/utils/validations.js
  28. +24,788 −16,930 package-lock.json
  29. +20 −21 package.json
  30. +37 −0 test/integration/{ → exports/simple}/sample-space.json
  31. BIN ...mages.ctfassets.net/n96dbk12ti8z/3zWbCFcZmV1WzzRYhVrd0O/3caa2d1f9d016cd0745ee74df1946137/logo.jpg
  32. BIN ...images.ctfassets.net/n96dbk12ti8z/t6Oc5OEaU8RMODFvQq96P/81fd6977c1b90cbe65b36dd9afac4861/logo.jpg
  33. +1,117 −0 test/integration/exports/with-assets/space-with-downloaded-assets.json
  34. +90 −36 test/integration/import-lib.test.js
  35. +27 −0 test/unit/__mocks__/fs.js
  36. +34 −25 test/unit/index.test.js
  37. +39 −7 test/unit/parseOptions.test.js
  38. +61 −61 test/unit/sort-entries.test.js
  39. +46 −25 test/unit/tasks/get-destination-data.test.js
  40. +47 −11 test/unit/tasks/push/assets.test.js
  41. +196 −47 test/unit/tasks/push/creation.test.js
  42. +74 −42 test/unit/tasks/push/publishing.test.js
  43. +83 −18 test/unit/tasks/push/push-to-space.test.js
  44. +1 −1 test/unit/transform/transform-space.test.js
  45. +17 −17 test/unit/transform/transformers.test.js
  46. +35 −0 test/unit/utils/headers.test.js
  47. +32 −8 test/unit/utils/schema.test.js
  48. +157 −157 test/unit/validations.test.js
13 changes: 0 additions & 13 deletions .babelrc

This file was deleted.

12 changes: 4 additions & 8 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -2,22 +2,17 @@ version: 2.1
jobs:
unit:
docker:
- image: circleci/node:12
- image: cimg/node:16.11
steps:
- checkout
- restore_cache:
key: dependency-cache-{{ checksum "package.json" }}
- run: npm install
- run: npm run build
- run: npm run test:unit
- run: bash <(curl -s https://codecov.io/bash)
- run:
name: Build Success
when: on_success
command: ./node_modules/.bin/codecov
integration:
docker:
- image: circleci/node:12
- image: cimg/node:16.11
steps:
- checkout
- restore_cache:
@@ -27,7 +22,7 @@ jobs:
- run: npm run test:integration
release:
docker:
- image: circleci/node:12
- image: cimg/node:16.11
steps:
- checkout
- run: npm install
@@ -40,6 +35,7 @@ workflows:
- unit
- integration
- release:
context: dev-workflows-release
requires:
- unit
- integration
13 changes: 0 additions & 13 deletions .dependabot/config.yml

This file was deleted.

24 changes: 10 additions & 14 deletions .eslintrc.js
Original file line number Diff line number Diff line change
@@ -1,15 +1,11 @@
module.exports = {
"parser": "babel-eslint",
"extends": [
"standard",
"plugin:jest/recommended"
],
"plugins": [
"standard",
"promise"
],
"rules": {
"jest/prefer-to-be-null": "warn",
"jest/prefer-to-be-undefined": "warn"
}
};
parser: 'babel-eslint',
extends: [
'standard',
'plugin:jest/recommended'
],
plugins: [
'standard',
'promise'
]
}
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* @contentful/team-dev-workflows
34 changes: 34 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
version: 2
registries:
npm-registry-registry-npmjs-org: # Only required if you use npm
type: npm-registry
url: https://registry.npmjs.org
token: "${{secrets.NPM_REGISTRY_REGISTRY_NPMJS_ORG_TOKEN}}"
updates:
- package-ecosystem: npm # Only required if you use npm
directory: "/"
schedule:
interval: daily
time: "00:00"
timezone: UTC
open-pull-requests-limit: 15
versioning-strategy: increase
ignore:
- dependency-name: "@types/node"
versions:
- ">=17.0.0"
commit-message:
prefix: build
include: scope
registries:
- npm-registry-registry-npmjs-org
- package-ecosystem: github-actions
directory: "/"
schedule:
interval: daily
time: "00:00"
timezone: UTC
open-pull-requests-limit: 15
commit-message:
prefix: build
include: scope
14 changes: 14 additions & 0 deletions .github/workflows/dependabot-approve-and-request-merge.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
name: "dependabot approve-and-request-merge"

on:
pull_request_target

jobs:
worker:
runs-on: ubuntu-latest

if: github.actor == 'dependabot[bot]'
steps:
- uses: contentful/dependabot-auto-merge@v1
with:
github_token: ${{ secrets.CF_ECOSYSTEM_BOT_GITHUB_TOKEN_AUTO_APPROVE }}
1 change: 1 addition & 0 deletions .nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
v16
45 changes: 38 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# Contentful import tool

[![npm](https://img.shields.io/npm/v/contentful-import.svg)](https://www.npmjs.com/package/contentful-import)
[![Build Status](https://travis-ci.org/contentful/contentful-import.svg?branch=master)](https://travis-ci.org/contentful/contentful-import)
[![codecov](https://codecov.io/gh/contentful/contentful-import/branch/master/graph/badge.svg)](https://codecov.io/gh/contentful/contentful-import)
[![CircleCI](https://circleci.com/gh/contentful/contentful-import.svg?style=shield)](https://circleci.com/gh/contentful/contentful-import/?branch=master)
[![Dependency Status](https://img.shields.io/david/contentful/contentful-import.svg)](https://david-dm.org/contentful/contentful-import)
[![devDependency Status](https://img.shields.io/david/dev/contentful/contentful-import.svg)](https://david-dm.org/contentful/contentful-import#info=devDependencies)

@@ -13,6 +12,7 @@
This library helps you to import files generated by [contentful-export](https://github.com/contentful/contentful-export) to a destination space.

## :exclamation: Usage as CLI

> We moved the CLI version of this tool into our [Contentful CLI](https://github.com/contentful/contentful-cli). This allows our users to use and install only one single CLI tool to get the full Contentful experience.
>
> Please have a look at the [Contentful CLI import command documentation](https://github.com/contentful/contentful-cli/tree/master/docs/space/import) to learn more about how to use this as command line tool.
@@ -93,51 +93,83 @@ contentfulImport(options)
### Basics

#### `spaceId` [string] [required]

ID of the space to import into

#### `environmentId` [string] [default: 'master']

ID of the environment in the destination space

#### `managementToken` [string] [required]

Contentful management API token for the space to be imported to

#### `contentFile` [string]

Path to JSON file that contains data to be import to your space

#### `content` [object]

Content to import. Needs to match the expected structure (See below)

### Filtering

#### `contentModelOnly` [boolean] [default: false]

Import content types only

#### `skipContentModel` [boolean] [default: false]

Skip importing of content types and locales

#### `skipLocales` [boolean] [default: false]

Skip importing of locales

#### `skipContentPublishing` [boolean] [default: false]

Skips content publishing. Creates content but does not publish it

## Assets

#### `uploadAssets` [boolean] [default: false]

Upload local asset files downloaded via the [downloadAssets](https://github.com/contentful/contentful-export#downloadassets-boolean) option of the export. Requires `assetsDirectory`

#### `assetsDirectory` [string]

Path to a directory with an asset export made using the [downloadAssets](https://github.com/contentful/contentful-export#downloadassets-boolean) option of the export. Requires `uploadAssets`

### Connection

#### `host` [string] [default: 'api.contentful.com']

The Management API host

#### `proxy` [string]

Proxy configuration in HTTP auth format: `host:port` or `user:password@host:port`

#### `rawProxy` [boolean]

Pass proxy config to Axios instead of creating a custom httpsAgent

#### `rateLimit` [number] [default: 7]

Maximum requests per second used for API requests

#### `headers` [object]

Additional headers to attach to the requests.

### Other

#### `errorLogFile` [string]

Full path to the error log file

#### `useVerboseRenderer` [boolean] [default: false]

Display progress in new lines instead of displaying a busy spinner and the status in the same line. Useful for CI.

## :rescue_worker_helmet: Troubleshooting
@@ -171,17 +203,16 @@ The data to import should be structured like this:

Note: `tags` are not available for all users. If you do not have access to this feature, any tags included in your import data will be skipped.


## :bulb: Importing to a space with existing content

- Both source space and destination space must share the same content model structure. In order to achieve that, please use [contentful-migration](https://www.npmjs.com/package/contentful-migration).
- Content transformations are also not supported, please use [contentful-migration](https://www.npmjs.com/package/contentful-migration).
- Entities existence are determined based on their ID:
* If an entity does not exist in the destination space, it will be created.
* If an entity already exists in the destination space, it will be updated.
- If an entity does not exist in the destination space, it will be created.
- If an entity already exists in the destination space, it will be updated.
- Publishing strategy:
* If an entity is in draft, it will be created as draft in the destination space.
* If an entity is published and has pending changes (updated) in the source space, it will be published with the latest changes in the destination space.
- If an entity is in draft, it will be created as draft in the destination space.
- If an entity is published and has pending changes (updated) in the source space, it will be published with the latest changes in the destination space.

## :warning: Limitations

16 changes: 16 additions & 0 deletions babel.config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"presets": [
[
"@babel/preset-env",
{
"targets": {
"node": "14"
}
}
]
],
"plugins": [
"@babel/plugin-proposal-object-rest-spread",
"add-module-exports"
]
}
4 changes: 2 additions & 2 deletions bin/contentful-import
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/usr/bin/env node
var runContentfulImport = require('../dist/index')
var usageParams = require('../dist/usageParams')
const runContentfulImport = require('../dist/index')
const usageParams = require('../dist/usageParams')
console.log('We moved the CLI version of this tool into our Contentful CLI.\nThis allows our users to use and install only one single CLI tool to get the full Contentful experience.\nFor more info please visit https://github.com/contentful/contentful-cli/tree/master/docs/space/import')
runContentfulImport(usageParams)
.then((result) => {
13 changes: 13 additions & 0 deletions catalog-info.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
name: contentful-import
description: |
Node module that uses the data provided by contentful-export to import it to contentful space
annotations:
circleci.com/project-slug: github/contentful/contentful-import
github.com/project-slug: contentful/contentful-import
spec:
type: cli
lifecycle: production
owner: group:team-dev-workflows
Loading