Compare commits

...

5 Commits

Author SHA1 Message Date
Anna “CyberTailor” 6111074d33
main: fixup for generators typing updates
continuous-integration/drone/push Build is passing Details
2024-05-02 00:49:21 +05:00
Anna “CyberTailor” 485b828b67
main: remove dead code 2024-05-02 00:44:37 +05:00
Anna “CyberTailor” b1a537cac5
docs: improve API documentation 2024-05-02 00:44:09 +05:00
Anna “CyberTailor” ca9f3c88eb
docs: reformat release notes 2024-05-01 23:18:21 +05:00
Anna “CyberTailor” 50230f9de3
docs: typo 2024-05-01 22:41:56 +05:00
24 changed files with 401 additions and 91 deletions

1
.gitignore vendored
View File

@ -7,7 +7,6 @@ __pycache__
*.pyo
docs/_build
docs/api
dist
Makefile

View File

@ -0,0 +1,9 @@
.. SPDX-FileCopyrightText: 2024 Anna <cyber@sysrq.in>
.. SPDX-License-Identifier: WTFPL
.. No warranty.
gentle.generators.python
========================
.. automodule:: gentle.generators.python
:members:

View File

@ -0,0 +1,14 @@
.. SPDX-FileCopyrightText: 2024 Anna <cyber@sysrq.in>
.. SPDX-License-Identifier: WTFPL
.. No warranty.
gentle.generators
=================
.. autosummary::
:toctree: generators
gentle.generators.python
.. automodule:: gentle.generators
:members:

View File

@ -0,0 +1,15 @@
.. SPDX-FileCopyrightText: 2024 Anna <cyber@sysrq.in>
.. SPDX-License-Identifier: WTFPL
.. No warranty.
gentle.metadata
===============
.. autosummary::
:toctree: metadata
gentle.metadata.types
gentle.metadata.utils
.. automodule:: gentle.metadata
:members:

14
docs/api/gentle.pms.rst Normal file
View File

@ -0,0 +1,14 @@
.. SPDX-FileCopyrightText: 2024 Anna <cyber@sysrq.in>
.. SPDX-License-Identifier: WTFPL
.. No warranty.
gentle.pms
==========
.. automodule:: gentle.pms
:members:
.. autosummary::
:toctree: pms
gentle.pms.portagepm

View File

@ -0,0 +1,9 @@
.. SPDX-FileCopyrightText: 2024 Anna <cyber@sysrq.in>
.. SPDX-License-Identifier: WTFPL
.. No warranty.
gentle.utils
============
.. automodule:: gentle.utils
:members:

View File

@ -0,0 +1,9 @@
.. SPDX-FileCopyrightText: 2024 Anna <cyber@sysrq.in>
.. SPDX-License-Identifier: WTFPL
.. No warranty.
gentle.metadata.types
=====================
.. automodule:: gentle.metadata.types
:members:

View File

@ -0,0 +1,9 @@
.. SPDX-FileCopyrightText: 2024 Anna <cyber@sysrq.in>
.. SPDX-License-Identifier: WTFPL
.. No warranty.
gentle.metadata.utils
=====================
.. automodule:: gentle.metadata.utils
:members:

View File

@ -0,0 +1,9 @@
.. SPDX-FileCopyrightText: 2024 Anna <cyber@sysrq.in>
.. SPDX-License-Identifier: WTFPL
.. No warranty.
gentle.pms.portagepm
====================
.. automodule:: gentle.pms.portagepm
:members:

View File

@ -21,6 +21,8 @@ release = '1.0.0'
extensions = [
'sphinx.ext.autosummary',
'sphinx.ext.autodoc',
'sphinx.ext.extlinks',
'sphinx.ext.intersphinx',
'sphinx_prompt',
]
@ -44,13 +46,23 @@ except ModuleNotFoundError:
root_doc = 'toc'
templates_path = ['_templates']
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
manpages_url = 'https://docs.sysrq.in/{path}'
autodoc_default_flags = [
"members",
"show-inheritance",
"inherited-members",
"undoc-members",
]
extlinks = {
'bug': ('https://bugs.sysrq.in/show_bug.cgi?id=%s', 'bug #%s'),
'gitweb': (f'https://git.sysrq.in/{project}/tree/%s', '%s'),
'pypi': ('https://pypi.org/project/%s/', '%s'),
}
intersphinx_mapping = {
'python': ('https://docs.python.org/3/', None),
}
autosummary_generate = False
autodoc_default_options = {
'show-inheritance': True,
'undoc-members': True,
'member-order': 'bysource',
}
# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
@ -61,6 +73,11 @@ html_theme_options = {
'globaltoc_maxdepth': 3,
'right_buttons': ['git-button.html'],
}
html_sidebars = {
'**': [
'globaltoc.html',
]
}
html_context = {
'git_repo_url': 'https://git.sysrq.in/gentle/about/',
}

View File

@ -7,7 +7,7 @@ Introduction
gentle is a :file:`metadata.xml` generator for Gentoo ebuilds.
If you want to learn about how to use gentle, check out the following resources:
If you want to learn how to use gentle, check out the following resources:
* :doc:`installation`
* :doc:`getting-started`
@ -27,9 +27,9 @@ If you find any bugs, please report them on `Bugzilla`_.
You might also like
-------------------
* `metagen`_: companion utility that generates basic package metadata.
- `metagen`_: companion utility that generates basic package metadata.
* `upstream-ontologist`_: distro-agnostic interface for finding metadata about
- `upstream-ontologist`_: distro-agnostic interface for finding metadata about
upstream software projects.
.. _metagen: https://cgit.gentoo.org/proj/metagen.git

View File

@ -1,4 +1,4 @@
.. SPDX-FileCopyrightText: 2023 Anna <cyber@sysrq.in>
.. SPDX-FileCopyrightText: 2023-2024 Anna <cyber@sysrq.in>
.. SPDX-License-Identifier: WTFPL
.. No warranty.
@ -7,6 +7,8 @@ API Reference
.. autosummary::
:toctree: api
:recursive:
gentle
gentle.generators
gentle.metadata
gentle.pms
gentle.utils

View File

@ -8,36 +8,109 @@ Release Notes
1.0.0
-----
* Drop support for Python 3.10.
* Fix metadata schema violation for URLs with whitespace in them
* [Ruby Gem] Support extracting metadata from Gemspec files, not just Gems
- **Gone**: Python 3.10 support.
- Fix metadata schema violation for URLs with whitespace in them.
*Generators changelog:*
- **Ruby Gem**:
- Support extracting metadata from Gemspec files, not just Gems.
*Tests changelog:*
- Monkey patch :pypi:`build` so it doesn't call Pip. With this change, ``--net``
option became obsolete and was removed.
- Add ``--with-ruby`` flag to enable tests that need Ruby.
0.4.1
-----
* New generator: Perl CPAN::Meta::Spec
* New generator: Ruby Gem
* [Docs] Fix Sphinx documentation
* [Tests] Add command-line flags to control tests selection
- **New generators**:
* Perl CPAN::Meta::Spec
* Ruby Gem
*Tests changelog:*
- Add command-line flags to control tests selection.
*Documentation changelog:*
- Fix configuration of Sphinx plugins.
0.4.0
-----
* New generator: Apache Maven POM
* New generator: Dart Pubspec
* New generator: GNU Autoconf
* New generator: NuGet
* New generator: PEAR/PECL
* New generator: Python Setuptools
* New generator: Python Wheel
* New CLI option to skip slow generators
* Add ``kde-invent`` remote-id
* Trim ".git" suffix when extracting remote-id
* Switch to the ``lxml`` library
* Don't write :file:`metadata.xml` if there are no changes
- **New generators**:
* Apache Maven POM
* Dart Pubspec
* GNU Autoconf
* NuGet
* PEAR/PECL
* Python Setuptools
* Python Wheel
- **New**: ``--quick`` option to skip slow generators.
- Add ``kde-invent`` to known remote-ids.
- Trim ".git" suffix when extracting remote-id.
- Don't write :file:`metadata.xml` if there are no changes.
*Dependencies introduced:*
* :pypi:`lxml`
* :pypi:`build` *(optional)*
0.3.1
-----
* Replace NIH metadata parser with Portage API-based parser
* Replace use of ``os.getlogin`` with a more reliable implementation
* Support setting ``EPREFIX`` via cli
- Replace NIH metadata parser with Portage API-based parser.
- Replace use of :py:func:`os.getlogin` with a more reliable implementation.
- Support setting ``EPREFIX`` via cli.
0.3.0
-----
- **New generators**:
* DOAP
* Haskell Hpack
* Python PKG-INFO
*Dependencies introduced:*
* :pypi:`pkginfo` *(optional)*
* :pypi:`rdflib` *(optional)*
*Documentation changelog:*
- Add Sphinx documentation.
0.2
---
- **New generators**:
* Bower
* Node.js NPM
* PHP Composer
* Rust Cargo
*Packaging:*
- Change Python dist-name from "gentle" to "gentle-mxml".
- Include tests in sdist.
0.1
---
- First release.

View File

@ -2,6 +2,8 @@
# SPDX-FileCopyrightText: 2022 Anna <cyber@sysrq.in>
# No warranty
""" Gentoo Metadata XML generator """
"""
Gentoo Metadata XML generator
"""
__version__ = "1.0.0"

View File

@ -2,8 +2,6 @@
# SPDX-FileCopyrightText: 2022-2023 Anna <cyber@sysrq.in>
# No warranty
# pylint: disable=unused-import
import argparse
import importlib.util
import logging
@ -13,7 +11,7 @@ from pathlib import Path
from tempfile import TemporaryDirectory
import gentle
from gentle.generators import AbstractGenerator, GeneratorClass
from gentle.generators import AbstractGenerator, GentleGenerator
from gentle.metadata import MetadataXML
import gentle.generators.autoconf
@ -36,8 +34,6 @@ import gentle.generators.shards
_HAS_PORTAGE = importlib.util.find_spec("portage") is not None
_HAS_BUILD = importlib.util.find_spec("build") is not None
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("main")
@ -54,10 +50,6 @@ def main() -> None:
if len(pm) == 0:
raise RuntimeError("No package manager installed. Aborting")
py_api = ["simple"]
if _HAS_BUILD:
py_api.insert(0, "wheel")
parser = argparse.ArgumentParser("gentle", description=gentle.__doc__)
parser.add_argument("ebuild", type=Path, help="path to the ebuild file")
parser.add_argument("--api", "-a", choices=pm, default=pm[0],
@ -82,11 +74,12 @@ def main() -> None:
try:
mxml = MetadataXML(mxml_file, parse_mxml)
except FileNotFoundError:
logger.error("Ebuild's metadata.xml file is missing, create it before running gentle")
logger.error("Ebuild's metadata.xml file is missing, create it "
"before running gentle")
sys.exit(1)
srcdir = src_unpack(args.ebuild, tmpdir)
cls: GeneratorClass
cls: GentleGenerator
for cls in AbstractGenerator.get_generator_subclasses():
generator = cls(srcdir)
if not generator.active:

View File

@ -2,7 +2,9 @@
# SPDX-FileCopyrightText: 2023 Anna <cyber@sysrq.in>
# No warranty
""" Generic generator routines """
"""
Generic generator interface.
"""
from abc import ABC, abstractmethod
from pathlib import Path
@ -10,35 +12,53 @@ from typing import Type
from gentle.metadata import MetadataXML
GentleGenerator = Type["AbstractGenerator"]
class AbstractGenerator(ABC):
""" Generic class for metadata generators. """
_subclasses: "list[GeneratorClass]" = []
"""
Generic class for metadata generators.
"""
_subclasses: list[GentleGenerator] = []
@classmethod
def get_generator_subclasses(cls) -> "list[GeneratorClass]":
def get_generator_subclasses(cls) -> list[GentleGenerator]:
"""
Get generators inheriting from this abstract class.
:returns: subclasses
"""
return cls._subclasses.copy()
@abstractmethod
def __init__(self, srcdir: Path):
...
"""
:param srcdir: path to unpacked sources
"""
def __init_subclass__(cls: "GeneratorClass", **kwargs: dict) -> None:
def __init_subclass__(cls, **kwargs: dict) -> None:
super().__init_subclass__(**kwargs)
AbstractGenerator._subclasses.append(cls)
@abstractmethod
def update_metadata_xml(self, mxml: MetadataXML) -> None:
...
"""
Update metadata object in place.
"""
@property
@abstractmethod
def active(self) -> bool:
...
"""
Whether generator works.
"""
@property
def slow(self) -> bool:
"""
Whether generator takes long time to finish.
"""
return False
GeneratorClass = Type[AbstractGenerator]

View File

@ -2,8 +2,11 @@
# SPDX-FileCopyrightText: 2023 Anna <cyber@sysrq.in>
# No warranty
""" Common data for Python generators """
"""
Common data for Python generators.
"""
#: Project URL names matching upstream bug tracker.
BUG_TRACKER_LABELS = [
"bug tracker",
"bug reports",
@ -12,19 +15,25 @@ BUG_TRACKER_LABELS = [
"issue tracker",
"issues",
"tracker",
"github: issues"
"github: issues",
]
#: Project URL names matching upstream changelog.
CHANGELOG_LABELS = [
"changelog",
"changes",
"release notes"
"release notes",
]
#: Project URL names matching upstream documentation.
DOCS_LABELS = [
"doc",
"docs",
"documentation",
"docs: rtd"
"docs: rtd",
]
#: Project URL names matching upstream repository or homepage.
HOME_REPO_LABELS = [
"code",
"source",
@ -34,5 +43,5 @@ HOME_REPO_LABELS = [
"home",
"homepage",
"github",
"github: repo"
"github: repo",
]

View File

@ -2,7 +2,9 @@
# SPDX-FileCopyrightText: 2023 Anna <cyber@sysrq.in>
# No warranty
""" Metadata routines """
"""
Metadata processing routines.
"""
import logging
from pathlib import Path
@ -16,13 +18,16 @@ logger = logging.getLogger("metadata")
class MetadataXML:
""" Modify :file:`metadata.xml` files """
"""
Modify :file:`metadata.xml` files.
"""
def __init__(self, xmlfile: Path, parser: Callable[[Path], Upstream]):
"""
:param xmlfile: Path to the :file:`metadata.xml` file
:param upstream: Pre-parsed :class:`Upstream` object
:param xmlfile: path to the :file:`metadata.xml` file
:param upstream: pre-parsed :class:`Upstream` object
"""
self.xmlfile: Path = xmlfile
self.xml: ET._ElementTree = ET.parse(self.xmlfile)
@ -41,7 +46,10 @@ class MetadataXML:
return url
def dump(self) -> None:
""" Write :file:`metadata.xml` file """
"""
Write :file:`metadata.xml` file.
"""
logger.info("Writing metadata.xml")
ET.indent(self.xml, space="\t", level=0)
self.xml.write(self.xmlfile,
@ -50,12 +58,22 @@ class MetadataXML:
encoding="UTF-8")
def dumps(self) -> str:
""" Convert the object to text """
"""
Convert the object to text.
:returns: XML data as text
"""
ET.indent(self.xml, space="\t", level=0)
return ET.tostring(self.xml.getroot(), encoding="unicode")
def add_upstream_maintainer(self, person: Person) -> None:
""" Add a person to the list of upstream maintainers """
"""
Add a person to the list of upstream maintainers.
:param person: upstrem maintainer
"""
if person in self.upstream.maintainers:
return
@ -67,7 +85,12 @@ class MetadataXML:
self.modified = True
def add_upstream_remote_id(self, remote_id: RemoteID) -> None:
""" Add an item to the list of remote ids """
"""
Add an item to the list of remote ids.
:param remote_id: new remote id
"""
for old_remote_id in self.upstream.remote_ids:
if remote_id.attr == old_remote_id.attr:
return
@ -80,7 +103,12 @@ class MetadataXML:
self.modified = True
def set_upstream_bugs_to(self, url: str) -> None:
""" Set upstream bugs-to URL """
"""
Set upstream bugs-to URL.
:param url: new URL
"""
url = self._encode_whitespace(url)
if self.upstream.bugs_to:
return
@ -95,7 +123,12 @@ class MetadataXML:
self.modified = True
def set_upstream_changelog(self, url: str) -> None:
""" Set upstream changelog URL """
"""
Set upstream changelog URL.
:param url: new URL
"""
url = self._encode_whitespace(url)
if self.upstream.changelog:
return
@ -110,7 +143,12 @@ class MetadataXML:
self.modified = True
def set_upstream_doc(self, url: str) -> None:
""" Set upstream documentation URL """
"""
Set upstream documentation URL.
:param url: new URL
"""
url = self._encode_whitespace(url)
if self.upstream.doc:
return

View File

@ -2,7 +2,9 @@
# SPDX-FileCopyrightText: 2023 Anna <cyber@sysrq.in>
# No warranty
""" Types for working with Gentoo package metadata """
"""
Types for working with Gentoo package metadata.
"""
from dataclasses import dataclass, field
from enum import Enum, auto
@ -11,19 +13,33 @@ import lxml.etree as ET
class MaintainerStatus(Enum):
""" Maintainer status enum """
"""
Maintainer status.
"""
#: Not specified.
NONE = auto()
#: Active.
ACTIVE = auto()
#: Inactive.
INACTIVE = auto()
@dataclass
class Person:
""" Representation of a person"""
"""
Representation of a person.
"""
#: Maintainer name.
name: str = field(default="", compare=False)
#: Maintainer email.
email: str = ""
#: Maintainer activity status.
status: MaintainerStatus = MaintainerStatus.NONE
def __str__(self) -> str:
@ -37,9 +53,13 @@ class Person:
def to_xml(self, attrib: dict | None = None) -> ET._Element:
"""
:param attrib: attributes for the ``<maintainer>`` tag
Make an XML ``<maintainer>`` tag.
:param attrib: attributes for the tag
:return: :file:`metadata.xml` respresentation of a person
"""
attrib = attrib or {}
match self.status:
case MaintainerStatus.ACTIVE:
@ -60,9 +80,14 @@ class Person:
@dataclass
class RemoteID:
""" Representation of a remote ID """
"""
Representation of a Remote ID.
"""
#: Site name.
attr: str
#: Package identificator on the site.
value: str
def __str__(self) -> str:
@ -70,8 +95,11 @@ class RemoteID:
def to_xml(self) -> ET._Element:
"""
Make an XML ``<remote-id>`` tag.
:return: :file:`metadata.xml` respresentation of a remote id
"""
remote_elem = ET.Element("remote-id", type=self.attr)
remote_elem.text = self.value
return remote_elem
@ -79,10 +107,21 @@ class RemoteID:
@dataclass
class Upstream:
""" Representation of upstream metadata """
"""
Representation of upstream metadata.
"""
#: Upstream maintainers.
maintainers: list[Person] = field(default_factory=list)
#: Upstream changelog.
changelog: str | None = None
#: Upstream documentation.
doc: str | None = None
#: Upstream bug tracker.
bugs_to: str | None = None
#: Upstream identificators on third-party sites.
remote_ids: list[RemoteID] = field(default_factory=list)

View File

@ -2,14 +2,18 @@
# SPDX-FileCopyrightText: 2023 Anna <cyber@sysrq.in>
# No warranty
""" Utilities for metadata generators """
"""
Utilities for metadata generators.
"""
import re
from gentle.metadata import Person, RemoteID
# Regular expression for matching "name <email>" pairs.
author_re = re.compile(r"(?P<name>.+?)\s*<(?P<email>.+?@.+?)>")
# Mapping of remote-ids to regular expressions matching them.
remote_ids = {
"bitbucket":
re.compile(r"^https?://bitbucket.org/(?P<v>[^\s/]+?/[^\s/]+?)([.]git)?(/.*)?$"),
@ -66,10 +70,12 @@ remote_ids = {
def extract_name_email(author: str) -> Person | None:
"""
Make a :class:`Person` object from a string.
Make a :py:class:`Person` object from a string.
:param author: string in the ``name <email>`` format
:returns: person object
>>> extract_name_email("Foo Bar <foobar@example.com>")
Person(name='Foo Bar', email='foobar@example.com', status=<MaintainerStatus.NONE: 1>)
>>> extract_name_email("Foo Bar") is None
@ -86,6 +92,8 @@ def extract_remote_id(url: str) -> RemoteID | None:
:param url: project's source repository
:returns: remote-id object
>>> extract_remote_id("https://pypi.org/project/foo-bar")
RemoteID(attr='pypi', value='foo-bar')
>>> extract_remote_id("https://example.com") is None

View File

@ -0,0 +1,7 @@
# SPDX-License-Identifier: WTFPL
# SPDX-FileCopyrightText: 2022-2023 Anna <cyber@sysrq.in>
# No warranty
"""
Functionality that depends on package management features.
"""

View File

@ -2,6 +2,10 @@
# SPDX-FileCopyrightText: 2022-2023 Anna <cyber@sysrq.in>
# No warranty
"""
Implementation of package management features with Portage API.
"""
import logging
import os
import pwd
@ -21,10 +25,12 @@ def src_unpack(ebuild: Path, tmpdir: str) -> Path:
"""
Unpack the sources using Portage.
:param ebuild: Path to the ebuild file
:param tmpdir: Temporary directory
:return: The value of ``${S}``
:param ebuild: path to the ebuild file
:param tmpdir: temporary directory
:return: value of ``${S}``
"""
ebuild = ebuild.resolve()
portdir = str(ebuild.parents[2])
@ -64,9 +70,11 @@ def src_unpack(ebuild: Path, tmpdir: str) -> Path:
def parse_mxml(xmlfile: Path) -> Upstream:
"""
Parse :file:`metadata.xml` files using Portage
Parse :file:`metadata.xml` files using Portage.
:param xmlfile: Path to the :file:`metadata.xml` file
:param xmlfile: path to the :file:`metadata.xml` file
:returns: upstream metadata information
"""
result = Upstream()

View File

@ -2,12 +2,22 @@
# SPDX-FileCopyrightText: 2023 Anna <cyber@sysrq.in>
# No warranty
""" Misc utilities """
"""
Utility functions and classes.
"""
import lxml.etree as ET
def stringify(element: ET._Element) -> str:
"""
Extract all text from the given XML element.
:param element: XML element object
:returns: text from the given element
"""
return "".join(
(text for text in element.itertext() if isinstance(text, str))
)

View File

@ -68,9 +68,6 @@ include = [
"docs/",
"tests/"
]
exclude = [
"docs/api/"
]
[tool.pytest.ini_options]
addopts = "--doctest-modules"