Skip to content

P
python-poetry
Commit 8e32aca5 by Sébastien Eustace
Options

Change the poetry.toml file for the new, standardized pyproject.toml

parent 785361c8
Showing with 199 additions and 135 deletions
.gitignore
......@@ -27,5 +27,5 @@ pip-log.txt
setup.cfg
setup.py
MANIFEST.in
poetry.lock
pyproject.lock
README.rst
CHANGELOG.md
......@@ -9,6 +9,7 @@
### Changed
- Changed the `poetry.toml` file for the new, standardized `pyproject.toml`.
- Dependencies of each package is now stored in `poetry.lock`.
- Improved TOML file management.
- Dependency resolver now respects the root package python version requirements.
......
README.md
......@@ -28,13 +28,13 @@ See `poet help completions` for full details, but the gist is as simple as using
```bash
# Bash
poetry completions bash > /etc/bash_completion.d/poetry.bash-completion
poetry completions bash > /etc/bash_completion.d/pyproject.bash-completion
# Bash (macOS/Homebrew)
poetry completions bash > $(brew --prefix)/etc/bash_completion.d/poetry.bash-completion
poetry completions bash > $(brew --prefix)/etc/bash_completion.d/pyproject.bash-completion
# Fish
poetry completions fish > ~/.config/fish/completions/poetry.fish
poetry completions fish > ~/.config/fish/completions/pyproject.fish
# Zsh
poetry completions zsh > ~/.zfunc/_poetry
......@@ -54,13 +54,13 @@ fpath+=~/.zfunc
## Introduction
`poetry` is a tool to handle dependencies installation, building and packaging of Python packages.
It only needs one file to do all of that: `poetry.toml`.
It only needs one file to do all of that: the new, [standardized](https://www.python.org/dev/peps/pep-0518/) `pyproject.toml`.
```toml
[package]
name = "pypoet"
[tool.poetry]
name = "my-package"
version = "0.1.0"
description = "Poet helps you declare, manage and install dependencies of Python projects, ensuring you have the right stack everywhere."
description = "The description of the package"
license = "MIT"
......@@ -70,17 +70,13 @@ authors = [
readme = 'README.md'
repository = "https://github.com/sdispater/poet"
homepage = "https://github.com/sdispater/poet"
keywords = ['packaging', 'poet']
include = ['poet/**/*', 'LICENSE']
python-versions = "~2.7 || ^3.2"
repository = "https://github.com/sdispater/poetry"
homepage = "https://github.com/sdispater/poetry"
keywords = ['packaging', 'poetry']
[dependencies]
[tool.pyproject.dependencies]
python = "~2.7 || ^3.2" # Compatible python versions must be declared here
toml = "^0.9"
requests = "^2.13"
semantic_version = "^2.6"
......@@ -90,13 +86,13 @@ wheel = "^0.29"
pip-tools = "^1.8.2"
cleo = { git = "https://github.com/sdispater/cleo.git", branch = "master" }
[dev-dependencies]
[tool.pyproject.dev-dependencies]
pytest = "^3.0"
pytest-cov = "^2.4"
coverage = "<4.0"
httpretty = "^0.8.14"
[scripts]
[tool.pyproject.scripts]
poet = 'poet:app.run'
```
......@@ -212,12 +208,12 @@ poetry new my-package
will create a folder as follows:
```text
my-project
├── poetry.toml
my-package
├── pyproject.toml
├── README.rst
├── my_project
├── my_package
└── __init__.py
├── tests
└── tests
├── __init__.py
└── test_my_package
```
......@@ -226,24 +222,24 @@ If you want to name your project differently than the folder, you can pass
the `--name` option:
```bash
poetry new my-folder --my-package
poetry new my-folder --name my-package
```
### install
The `install` command reads the `poetry.toml` file from the current directory, resolves the dependencies,
The `install` command reads the `pyproject.toml` file from the current directory, resolves the dependencies,
and installs them.
```bash
poetry install
```
If there is a `poetry.lock` file in the current directory,
If there is a `pyproject.lock` file in the current directory,
it will use the exact versions from there instead of resolving them.
This ensures that everyone using the library will get the same versions of the dependencies.
If there is no `poetry.lock` file, Poetry will create one after dependency resolution.
If there is no `pyproject.lock` file, Poetry will create one after dependency resolution.
You can specify to the command that you do not want the development dependencies installed by passing
the `--no-dev` option.
......@@ -267,14 +263,14 @@ poetry install -f mysql -f pgsql
### update
In order to get the latest versions of the dependencies and to update the `poetry.lock` file,
In order to get the latest versions of the dependencies and to update the `pyproject.lock` file,
you should use the `update` command.
```bash
poetry update
```
This will resolve all dependencies of the project and write the exact versions into `poetry.lock`.
This will resolve all dependencies of the project and write the exact versions into `pyproject.lock`.
If you just want to update a few packages and not all, you can list them as such:
......@@ -284,12 +280,11 @@ poetry update requests toml
#### Options
* `--no-progress`: Removes the progress display that can mess with some terminals or scripts which don't handle backspace characters.
* `--dry-run` : Outputs the operations but will not execute anything (implicitly enables --verbose).
### add
The `add` command adds required packages to your `poetry.toml` and installs them.
The `add` command adds required packages to your `pyproject.toml` and installs them.
If you do not specify a version constraint,
poetry will choose a suitable one based on the available package versions.
......@@ -384,16 +379,16 @@ poetry search requests pendulum
### lock
This command locks (without installing) the dependencies specified in `poetry.toml`.
This command locks (without installing) the dependencies specified in `pyproject.toml`.
```bash
poetry lock
```
## The `poetry.toml` file
## The `pyproject.toml` file
A `poetry.toml` file is composed of multiple sections.
A `pyproject.toml` file is composed of multiple sections.
### package
......@@ -519,7 +514,7 @@ Only the name and a version string are required in this case.
requests = "^2.13.0"
```
If you want to use a private repository, you can add it to your `poetry.toml` file, like so:
If you want to use a private repository, you can add it to your `pyproject.toml` file, like so:
```toml
[[source]]
......@@ -684,5 +679,5 @@ To match the example in the setuptools documentation, you would use the followin
## Resources
* [Official Website](https://poetry.eustace.io)
* [Official Website](https://pyproject.eustace.io)
* [Issue Tracker](https://github.com/sdispater/poetry/issues)
poetry/console/commands/add.py
......@@ -35,11 +35,12 @@ If you do not specify a version constraint, poetry will choose a suitable one ba
if is_dev:
section = 'dev-dependencies'
original_content = self.poetry.locker.original.read()
content = self.poetry.locker.original.read()
original_content = self.poetry.file.read()
content = self.poetry.file.read()
poetry_content = content['tool']['poetry']
for name in packages:
for key in content[section]:
for key in poetry_content[section]:
if key.lower() == name.lower():
raise ValueError(f'Package {name} is already present')
......@@ -52,10 +53,10 @@ If you do not specify a version constraint, poetry will choose a suitable one ba
parser.parse_constraints(constraint)
for name, constraint in requirements.items():
content[section][name] = constraint
poetry_content[section][name] = constraint
# Write new content
self.poetry.locker.original.write(content)
self.poetry.file.write(content)
# Cosmetic new line
self.line('')
......@@ -77,7 +78,7 @@ If you do not specify a version constraint, poetry will choose a suitable one ba
try:
status = installer.run()
except Exception:
self.poetry.locker.original.write(original_content)
self.poetry.file.write(original_content)
raise
......@@ -90,7 +91,7 @@ If you do not specify a version constraint, poetry will choose a suitable one ba
'to its original content.'
)
self.poetry.locker.original.write(original_content)
self.poetry.file.write(original_content)
return status
......
poetry/console/commands/remove.py
......@@ -23,8 +23,9 @@ list of installed packages
packages = self.argument('packages')
is_dev = self.option('dev')
original_content = self.poetry.locker.original.read()
content = self.poetry.locker.original.read()
original_content = self.poetry.file.read()
content = self.poetry.file.read()
poetry_content = content['tool']['poetry']
section = 'dependencies'
if is_dev:
section = 'dev-dependencies'
......@@ -33,20 +34,20 @@ list of installed packages
requirements = {}
for name in packages:
found = False
for key in content[section]:
for key in poetry_content[section]:
if key.lower() == name.lower():
found = True
requirements[name] = content[section][name]
requirements[name] = poetry_content[section][name]
break
if not found:
raise ValueError(f'Package {name} not found')
for key in requirements:
del content[section][key]
del poetry_content[section][key]
# Write the new content back
self.poetry.locker.original.write(content)
self.poetry.file.write(content)
# Update packages
self.reset_poetry()
......@@ -65,7 +66,7 @@ list of installed packages
try:
status = installer.run()
except Exception:
self.poetry.locker.original.write(original_content)
self.poetry.file.write(original_content)
raise
......@@ -78,6 +79,6 @@ list of installed packages
'to its original content.'
)
self.poetry.locker.original.write(original_content)
self.poetry.file.write(original_content)
return status
poetry/installation/installer.py
......@@ -143,7 +143,7 @@ class Installer:
self._io.writeln(
'<warning>'
'Warning: The lock file is not up to date with '
'the latest changes in composer.json. '
'the latest changes in pyproject.toml. '
'You may be getting outdated dependencies. '
'Run update to update them.'
'</warning>'
......
poetry/layouts/layout.py
......@@ -90,25 +90,31 @@ class Layout(object):
def _write_poetry(self, path):
output = {
'package': {
'tool': {
'poetry': {
'name': self._project,
'version': self._version,
'authors': [self._author],
}
}
}
content = toml.dumps(output, preserve=True)
output = {
'tool': {
'poetry': {
'dependencies': {},
'dev-dependencies': {
'pytest': '^3.4'
}
}
}
}
content += '\n' + toml.dumps(output, preserve=True)
poetry = path / 'poetry.toml'
poetry = path / 'pyproject.toml'
with poetry.open('w') as f:
f.write(content)
......
poetry/packages/locker.py
......@@ -16,22 +16,19 @@ class Locker:
'name',
'version',
'python-versions',
'platform',
'dependencies',
'dev-dependencies',
'source',
]
def __init__(self, lock: Path, original: Path):
def __init__(self, lock: Path, local_config: dict):
self._lock = TomlFile(lock)
self._original = TomlFile(original)
self._local_config = local_config
self._lock_data = None
self._content_hash = self._get_content_hash()
@property
def original(self) -> TomlFile:
return self._original
@property
def lock(self) -> TomlFile:
return self._lock
......@@ -55,10 +52,11 @@ class Locker:
"""
Checks whether the lock file is still up to date with the current hash.
"""
lock = self._lock.read()
lock = self._lock.read(True)
metadata = lock.get('metadata', {})
if 'content-hash' in lock:
return self._content_hash == lock['content-hash']
if 'content-hash' in metadata:
return self._content_hash == lock['metadata']['content-hash']
return False
......@@ -140,16 +138,11 @@ class Locker:
"""
Returns the sha256 hash of the sorted content of the composer file.
"""
content = self._original.read()
content = self._local_config
relevant_content = {}
package = content['package']
for key in ['name', 'version', 'python-versions', 'platform']:
relevant_content[key] = package.get(key, '')
for key in ['dependencies', 'dev-dependencies']:
relevant_content[key] = content[key]
for key in self._relevant_keys:
relevant_content[key] = content.get(key)
content_hash = sha256(
json.dumps(relevant_content, sort_keys=True).encode()
......@@ -163,7 +156,7 @@ class Locker:
'No lockfile found. Unable to read locked packages'
)
return self._lock.read()
return self._lock.read(True)
def _lock_packages(self,
packages: List['poetry.packages.Package']) -> list:
......
poetry/poetry.py
......@@ -13,9 +13,11 @@ class Poetry:
VERSION = '0.2.0'
def __init__(self,
file: Path,
config: dict,
package: Package,
locker: Locker):
self._file = TomlFile(file)
self._package = package
self._config = config
self._locker = locker
......@@ -29,6 +31,10 @@ class Poetry:
self._pool.add_repository(PyPiRepository())
@property
def file(self):
return self._file
@property
def package(self) -> Package:
return self._package
......@@ -46,37 +52,42 @@ class Poetry:
@classmethod
def create(cls, cwd) -> 'Poetry':
poetry_file = Path(cwd) / 'poetry.toml'
poetry_file = Path(cwd) / 'pyproject.toml'
if not poetry_file.exists():
raise RuntimeError(
f'Poetry could not find a poetry.json file in {cwd}'
f'Poetry could not find a pyproject.toml file in {cwd}'
)
# TODO: validate file content
local_config = TomlFile(poetry_file.as_posix()).read()
local_config = TomlFile(poetry_file.as_posix()).read(True)
if 'tool' not in local_config or 'poetry' not in local_config['tool']:
raise RuntimeError(
f'[tool.poetry] section not found in {poetry_file.name}'
)
local_config = local_config['tool']['poetry']
# Load package
package_config = local_config['package']
name = package_config['name']
pretty_version = package_config['version']
name = local_config['name']
pretty_version = local_config['version']
version = normalize_version(pretty_version)
package = Package(name, version, pretty_version)
if 'python-versions' in package_config:
package.python_versions = package_config['python-versions']
if 'platform' in package_config:
package.platform = package_config['platform']
if 'platform' in local_config:
package.platform = local_config['platform']
if 'dependencies' in local_config:
for name, constraint in local_config['dependencies'].items():
if name.lower() == 'python':
package.python_versions = constraint
continue
package.add_dependency(name, constraint)
if 'dev-dependencies' in local_config:
for name, constraint in local_config['dev-dependencies'].items():
package.add_dependency(name, constraint, category='dev')
locker = Locker(poetry_file.with_suffix('.lock'), poetry_file)
locker = Locker(poetry_file.with_suffix('.lock'), local_config)
return cls(local_config, package, locker)
return cls(poetry_file, local_config, package, locker)
poetry/toml/cascadedict.py
......@@ -5,8 +5,10 @@ from . import raw
class CascadeDict:
"""
A dict-like object made up of one or more other dict-like objects where querying for an item cascade-gets
it from all the internal dicts in order of their listing, and setting an item sets it on the first dict listed.
A dict-like object made up of one or more other dict-like objects
where querying for an item cascade-gets it from all the internal dicts
in order of their listing, and setting an item
sets it on the first dict listed.
"""
def __init__(self, *internal_dicts):
......@@ -17,7 +19,7 @@ class CascadeDict:
"""
Returns another instance with one more dict cascaded at the end.
"""
return CascadeDict(self._internal_dicts, one_more_dict,)
return CascadeDict(*self._internal_dicts, one_more_dict)
def __getitem__(self, item):
for d in self._internal_dicts:
......
poetry/toml/prettify/elements/table.py
......@@ -109,9 +109,13 @@ class TableElement(abstracttable.AbstractTable):
end = len(tuple(self._sub_elements))
self._sub_elements = self.sub_elements[:begin] + self.sub_elements[end:]
@property
def value(self):
return self
def __eq__(self, other):
return self.primitive_value == other
def __iter__(self):
return iter(self.keys())
......
poetry/toml/structurer.py
......@@ -17,30 +17,29 @@ class NamedDict(dict):
"""
key can be an Name instance.
When key is a path in the form of an Name instance, all the parents and grandparents of the value are
created along the way as instances of NamedDict. If the parent of the value exists, it is replaced with a
CascadeDict() that cascades the old parent value with a new NamedDict that contains the given child name
and value.
When key is a path in the form of an Name instance,
all the parents and grandparents of the value are
created along the way as instances of NamedDict.
If the parent of the value exists, it is replaced with a
CascadeDict() that cascades the old parent value
with a new NamedDict that contains the given child name and value.
"""
if isinstance(key, toplevels.Name):
if len(key.sub_names) == 1:
name = key.sub_names[0]
if name in self:
self[name] = CascadeDict(self[name], value)
obj = self
for i, name in enumerate(key.sub_names):
if name in obj:
if i == len(key.sub_names) - 1:
obj[name] = CascadeDict(obj[name], value)
else:
self[name] = value
elif len(key.sub_names) > 1:
name = key.sub_names[0]
rest_of_key = key.drop(1)
if name in self:
named_dict = NamedDict()
named_dict[rest_of_key] = value
self[name] = CascadeDict(self[name], named_dict)
obj[name] = CascadeDict(NamedDict(), obj[name])
else:
self[name] = NamedDict()
self[name][rest_of_key] = value
if i == len(key.sub_names) - 1:
obj[name] = value
else:
obj[name] = NamedDict()
obj = obj[name]
else:
return dict.__setitem__(self, key, value)
......@@ -62,7 +61,6 @@ class NamedDict(dict):
self[key] = [value]
def __getitem__(self, item):
if isinstance(item, toplevels.Name):
d = self
for name in item.sub_names:
......@@ -71,13 +69,15 @@ class NamedDict(dict):
else:
return dict.__getitem__(self, item)
def __eq__(self, other):
return dict.__eq__(self, other)
def structure(table_toplevels):
"""
Accepts an ordered sequence of TopLevel instances and returns a navigable object structure representation of the
TOML file.
Accepts an ordered sequence of TopLevel instances and returns a navigable
object structure representation of the TOML file.
"""
table_toplevels = tuple(table_toplevels)
obj = NamedDict()
......
poetry/utils/toml_file.py
......@@ -16,7 +16,10 @@ class TomlFile:
def path(self):
return self._path
def read(self) -> dict:
def read(self, raw=False) -> dict:
if raw:
return toml.loads(self._path.read_text())
return loads(self._path.read_text())
def write(self, data) -> None:
......
poetry.toml → pyproject.toml
[package]
[tool.poetry]
name = "poetry"
version = "0.2.0"
description = "Python dependency management and packaging made easy."
python-versions = "^3.6"
license = "MIT"
authors = [
"Sébastien Eustace <sebastien@eustace.io>"
]
license = "MIT"
readme = "README.md"
......@@ -19,19 +15,14 @@ documentation = "https://poetry.eustace.io/docs"
keywords = ["packaging", "dependency", "poetry"]
include = ['poetry/**/*', 'LICENSE']
[dependencies]
# Requirements
[tool.poetry.dependencies]
python = "^3.6"
cleo = "^0.6"
requests = "^2.18"
toml = "^0.9"
cachy = "^0.1.0"
pip-tools = "^1.11"
toml = "^0.9.4"
[dev-dependencies]
[tool.poetry.dev-dependencies]
pytest = "~3.4"
[scripts]
poetry = 'poetry:console.run'
tests/toml/test_named_dict.py 0 → 100644
from poetry.toml.structurer import NamedDict
from poetry.toml.toplevels import Name
from poetry.toml.prettify.elements.table import TableElement
def test_named_dict():
d = NamedDict()
d[Name(('root', 'sub1', 'sub2'))] = 'foo'
assert d['root']['sub1']['sub2'] == 'foo'
assert d['root'] == {
'sub1': {
'sub2': 'foo'
}
}
d = NamedDict()
d[Name(('root', 'ns'))] = TableElement({})
d[Name(('root', 'ns', 'sub2'))] = TableElement({})
d[Name(('root', 'ns', 'sub3'))] = TableElement({})
assert d['root']['ns']['sub2'] == {}
assert d['root']['ns']['sub3'] == {}
tests/toml/test_toml.py
from poetry.toml import dumps
from poetry.toml import loads
from poetry.toml.prettify.errors import TOMLError
from poetry.toml.prettify.errors import DuplicateKeysError
from poetry.toml.prettify.errors import DuplicateTablesError
from poetry.toml.prettify.errors import InvalidTOMLFileError
def test_loading_toml_without_trailing_newline():
......@@ -91,7 +87,7 @@ cwd = "./handlers"
REDIS_PASSWORD = "MYPASSWORD"
#REDIS_PASSWORD = ""
"""
print(f.dumps())
assert expected == f.dumps()
......
tests/utils/fixtures/pyproject.toml 0 → 100644
[tool.poetry]
name = "poetry"
version = "0.2.0"
description = "Python dependency management and packaging made easy."
authors = [
"Sébastien Eustace <sebastien@eustace.io>"
]
license = "MIT"
readme = "README.md"
homepage = "https://poetry.eustace.io/"
repository = "https://github.com/sdispater/poet"
documentation = "https://poetry.eustace.io/docs"
keywords = ["packaging", "dependency", "poetry"]
# Requirements
[tool.poetry.dependencies]
python = "^3.6"
cleo = "^0.6"
requests = "^2.18"
toml = "^0.9"
cachy = "^0.1.0"
pip-tools = "^1.11"
[tool.poetry.dev-dependencies]
pytest = "~3.4"
tests/utils/test_toml_file.py
......@@ -48,3 +48,12 @@ def test_toml_file(fixture):
assert len(apple) == 3
banana = fruits[1]
assert len(banana['variety'][0]['points']) == 3
def test_pyproject_parsing(fixture):
f = TomlFile(fixture.with_name('pyproject.toml'))
content = f.read()
assert 'dependencies' in content['tool']['poetry']
assert content['tool']['poetry']['dependencies']['python'] == '^3.6'
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment