How to use the lmdb.MemoryError function in lmdb

To help you get started, we’ve selected a few lmdb examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github ArchiveTeam / grab-site / libgrabsite / dupes.py View on Github external
# but sometimes needs to be even lower.
		for map_size in (1024**4, 2**31 - 1, 2**30 - 1, 2**29 - 1, 2**28 - 1, 2**27 - 1):
			try:
				self._env = lmdb.open(
					filename,
					# Can't use writemap=True on OS X because it does not fully support sparse files
					# https://acid.readthedocs.org/en/latest/engines.html
					#
					# Don't use writemap=True elsewhere to avoid creating enormous sparse files
					# that will inevitably get copied without hole-skipping and fill up a disk.
					writemap=False,
					sync=False,
					metasync=False,
					# http://lmdb.readthedocs.org/en/release/#lmdb.Environment
					map_size=map_size)
			except (OverflowError, lmdb.MemoryError, lmdb.Error) as e:
				last_error = e
			else:
				print(f"Created lmdb db with map_size={map_size}")
				last_error = None
				break
		if last_error is not None:
			raise last_error