etcd3-py

Python client for etcd v3 (Using gRPC-JSON-Gateway)

• Free software: Apache Software License 2.0
• Source Code: https://github.com/Revolution1/etcd3-py
• Documentation: https://etcd3-py.readthedocs.io.
• etcd version required: v3.2.2+

Notice: The authentication header through gRPC-JSON-Gateway only supported in etcd v3.3.0+

Features

• [x] Support python2.7 and python3.5+ (aiohttp requires python3.5.2+)
• [x] Sync client based on requests
• [x] Async client based on aiohttp
• [x] TLS Connection
• [x] support APIs
  • [x] Auth
  • [x] KV
  • [x] Watch
  • [x] Cluster
  • [x] Lease
  • [x] Lock
  • [x] Maintenance
  • [x] Extra APIs
• [x] stateful utilities
  • [x] Watch
  • [x] Lease
  • [x] Transaction
  • [x] Lock

Quick Start

Install

$ pip install etcd3-py

Sync Client

>>> from etcd3 import Client
>>> client = Client('127.0.0.1', 2379, cert=(CERT_PATH, KEY_PATH), verify=CA_PATH)
>>> client.version()
EtcdVersion(etcdserver='3.3.0-rc.4', etcdcluster='3.3.0')
>>> client.put('foo', 'bar')
etcdserverpbPutResponse(header=etcdserverpbResponseHeader(cluster_id=11588568905070377092, member_id=128088275939295631, revision=15433, raft_term=4))
>>> client.range('foo').kvs
[mvccpbKeyValue(key=b'foo', create_revision=15429, mod_revision=15433, version=5, value=b'bar')]

Async Client (Python3.5+)

>>> import asyncio
>>> from etcd3 import AioClient
>>> client = AioClient('127.0.0.1', 2379)
>>> async def getFoo():
...     await client.put('foo', 'bar')
...     r = await client.range('foo')
...     print('key:', r.kvs[0].key, 'value:', r.kvs[0].value)
>>> loop = asyncio.get_event_loop()
>>> loop.run_until_complete(getFoo())
key: b'foo' value: b'bar'

Transaction Util

>>> from etcd3 import Client
>>> txn = Client().Txn()
>>> txn.compare(txn.key('foo').value == 'bar')
>>> txn.success(txn.put('foo', 'bra'))
>>> txn.commit()
etcdserverpbTxnResponse(header=etcdserverpbResponseHeader(cluster_id=11588568905070377092, member_id=128088275939295631, revision=15656, raft_term=4), succeeded=True, responses=[etcdserverpbResponseOp(response_put=etcdserverpbPutResponse(header=etcdserverpbResponseHeader(revision=15656)))])

Lease Util

>>> from etcd3 import Client
>>> client = Client()
>>> with client.Lease(ttl=5) as lease:
...     client.put('foo', 'bar', lease=lease.ID)
...     client.put('fizz', 'buzz', lease=lease.ID)
...     r = lease.time_to_live(keys=True)
...     assert set(r.keys) == {b'foo', b'fizz'}
...     assert lease.alive()

Watch Util

>>> from etcd3 import Client
>>> client = Client()
>>> watcher = c.Watcher(all=True, progress_notify=True, prev_kv=True)
>>> w.onEvent('f.*', lambda e: print(e.key, e.value))
>>> w.runDaemon()
>>> # etcdctl put foo bar
>>> # etcdctl put foz bar
b'foo' b'bar'
b'foz' b'bar'
>>> w.stop()

Lock Util

>>> import time
>>> from threading import Thread
>>> from etcd3 import Client
>>> client = Client()
>>> name = 'lock_name'
>>> def user1():
...     with client.Lock(name, lock_ttl=5):
...         print('user1 got the lock')
...         time.sleep(5)
...         print('user1 releasing the lock')
>>> def user2():
...     with client.Lock(name, lock_ttl=5):
...         print('user2 got the lock')
...         time.sleep(5)
...         print('user2 releasing the lock')
>>> t1 = Thread(target=user1, daemon=True)
>>> t2 = Thread(target=user2, daemon=True)
>>> t1.start()
>>> t2.start()
>>> t1.join()
>>> t2.join()
user1 got the lock
user1 releasing the lock
user2 got the lock
user2 releasing the lock

Start a single-node etcd using docker

export NODE1=0.0.0.0
export ETCD_VER=v3.3
docker run -d \
-p 2379:2379 \
-p 2380:2380 \
--volume=/tmp/etcd3-data:/etcd-data \
--name etcd3 quay.io/coreos/etcd:$ETCD_VER \
/usr/local/bin/etcd \
--data-dir=/etcd-data --name node1 \
--initial-advertise-peer-urls http://${NODE1}:2380 --listen-peer-urls http://${NODE1}:2380 \
--advertise-client-urls http://${NODE1}:2379 --listen-client-urls http://${NODE1}:2379 \
--initial-cluster node1=http://${NODE1}:2380

FAQ

Q: authentication seems not working? Try calling api of a auth-enabled etcd server returned error "ErrUserEmpty error:'etcdserver: user name is empty'"

A: Take a look at #41, currently etcd3-py dose not authenticate automatically, you need to call client.auth() by yourself.

TODO

• [ ] human friendly middle level apis
• [ ] able to expose json or raw response to user
• [ ] add election api
• [ ] benchmark
• [ ] python-etcd(etcd v2) compatible client
• [ ] etcd browser
• [ ] support etcd v3.4.x

History

0.1.6 (2019-05-9)

• merge pull request #90 Fix lease util keeping problems
• merge pull request #89 Add range end and lease to txn
• merge pull request #87 Add handel null value as gogoproto does while modelizing response data
• merge pull request #82 Fix watch util issue #18 and #78
• merge pull request #79 Improve etcd comapabitity of multiple versions
• merge pull request #51 Add a base EtcdModel to all dynamic created model
• merge pull request #42 Improve etcd comapabitity of multiple versions

0.1.5 (2018-07-4)

• merge pull request #34 enum34 only where it's needed

0.1.4 (2018-03-30)

• better code quality
• support etcd v3.2.2+

0.1.3 (2018-03-21)

• finished lock util

0.1.2 (2018-03-20)

• Add more test
• Add watcher, transaction and lease util

You can try it at dev environment

0.1.0 (2018-03-19)

• Implemented all APIs of etcd3's gRPC-JSON-Gateway
• Stateful utils (Watcher Lease Lock Transaction) are in progress