PBA Downloader
- Wolfgang Glas
Owned by Wolfgang Glas
svn URL: svn+ssh://pba@svn.clazzes.org/svn/pba/trunk/pba-download
Package name: python3-pba-download
Description
The PBA downloader fetches one or more packages from a list of yum or deb repositories without using any data persisted on the host system.
It may be fed with a config file describing a large number of packages to be downloaded to one or more subdirectories of the current directory.
DEB repositories
All-in-one cmdline
This variant allows to download multiple packages using one or more repositories specified by the -r <debline> option. The debian repository line must not contain components, which are taken from the release file of the repository if not explicitly specified.
An architecture must be specified using the option -a <arch>.
Download debian packages by specifying the repository on the cmdline.
$ pba-download -r 'deb http://ftp.at.debian.org/debian stretch' -a amd64 libc6 libc6-dev 2018-07-04 22:32:37,905 INFO Loading URL [http://ftp.at.debian.org/debian/dists/stretch/InRelease]... 2018-07-04 22:32:37,976 WARNING File [dists/stretch/InRelease] not found, using Release and Release.gpg 2018-07-04 22:32:37,977 INFO Loading URL [http://ftp.at.debian.org/debian/dists/stretch/Release]... 2018-07-04 22:32:38,174 INFO Loading URL [http://ftp.at.debian.org/debian/dists/stretch/Release.gpg]... 2018-07-04 22:32:38,257 INFO Checking GPG signature on [dists/stretch/Release] 2018-07-04 22:32:38,296 INFO gpgv: Signatur vom Sam 10 Mär 2018 11:22:39 CET 2018-07-04 22:32:38,296 INFO gpgv: mittels RSA-Schlüssel A1BD8E9D78F7FE5C3E65D8AF8B48AD6246925553 2018-07-04 22:32:38,297 INFO gpgv: Korrekte Signatur von "Debian Archive Automatic Signing Key (7.0/wheezy) <ftpmaster@debian.org>" 2018-07-04 22:32:38,297 INFO gpgv: Signatur vom Sam 10 Mär 2018 11:22:40 CET 2018-07-04 22:32:38,297 INFO gpgv: mittels RSA-Schlüssel 126C0D24BD8A2942CC7DF8AC7638D0442B90D010 2018-07-04 22:32:38,298 INFO gpgv: Korrekte Signatur von "Debian Archive Automatic Signing Key (8/jessie) <ftpmaster@debian.org>" 2018-07-04 22:32:38,304 INFO gpgv: Signatur vom Sam 10 Mär 2018 11:25:44 CET 2018-07-04 22:32:38,304 INFO gpgv: mittels RSA-Schlüssel 067E3C456BAE240ACEE88F6FEF0F382A1A7B6500 2018-07-04 22:32:38,304 INFO gpgv: Korrekte Signatur von "Debian Stable Release Key (9/stretch) <debian-release@lists.debian.org>" 2018-07-04 22:32:38,304 INFO GPG signature on [dists/stretch/Release] is OK. 2018-07-04 22:32:38,306 WARNING Ignoring unknown Release tag [Changelogs: http://metadata.ftp-master.debian.org/changelogs/@CHANGEPATH@_changelog] of resource [http://ftp.at.debian.org/debian/dists/stretch/Release]. 2018-07-04 22:32:38,307 WARNING Ignoring unknown Release tag [Acquire-By-Hash: yes] of resource [http://ftp.at.debian.org/debian/dists/stretch/Release]. 2018-07-04 22:32:38,377 INFO Taking components ['main', 'contrib', 'non-free'] from [http://ftp.at.debian.org/debian/dists/stretch/Release] 2018-07-04 22:32:38,378 INFO Loading URL [http://ftp.at.debian.org/debian/dists/stretch/main/binary-amd64/Packages.gz]... 2018-07-04 22:32:58,486 INFO Successfully loaded [50838] package headers from [http://ftp.at.debian.org/debian/dists/stretch/main/binary-amd64/Packages.gz]. 2018-07-04 22:32:58,487 INFO Loading URL [http://ftp.at.debian.org/debian/dists/stretch/contrib/binary-amd64/Packages.gz]... 2018-07-04 22:32:58,674 INFO Successfully loaded [303] package headers from [http://ftp.at.debian.org/debian/dists/stretch/contrib/binary-amd64/Packages.gz]. 2018-07-04 22:32:58,675 INFO Loading URL [http://ftp.at.debian.org/debian/dists/stretch/non-free/binary-amd64/Packages.gz]... 2018-07-04 22:32:58,908 INFO Successfully loaded [517] package headers from [http://ftp.at.debian.org/debian/dists/stretch/non-free/binary-amd64/Packages.gz]. 2018-07-04 22:32:58,908 INFO Found package [libc6] in repository [stretch/main] 2018-07-04 22:32:58,908 INFO Loading URL [http://ftp.at.debian.org/debian/pool/main/g/glibc/libc6_2.24-11+deb9u3_amd64.deb]... 2018-07-04 22:33:01,733 INFO Saved file [libc6_2.24-11+deb9u3_amd64.deb] with checksum [e57b3e24ea79fcdb46549d4ed2b95bb9657f21bba60ed5d9136d5b7112500084] 2018-07-04 22:33:01,734 INFO Setting modification time of file [libc6_2.24-11+deb9u3_amd64.deb] to [2018-02-15 04:41:07] 2018-07-04 22:33:01,734 INFO Found package [libc6-dev] in repository [stretch/main] 2018-07-04 22:33:01,735 INFO Loading URL [http://ftp.at.debian.org/debian/pool/main/g/glibc/libc6-dev_2.24-11+deb9u3_amd64.deb]... 2018-07-04 22:33:04,207 INFO Saved file [libc6-dev_2.24-11+deb9u3_amd64.deb] with checksum [8bdebd7bc1fc4138e0181821a1fe1fb576cbae241f03a31ab3c6cfc3a9875dc6] 2018-07-04 22:33:04,207 INFO Setting modification time of file [libc6-dev_2.24-11+deb9u3_amd64.deb] to [2018-02-15 04:41:07]
Download using a config file
You may create a config file, which contains multiple package names and multiple repository lines.
deb-packages.conf config file for large-scale debian download
[deb] osgi-runner-2=deb https://deb.clazzes.org/any any-osgi-runner-2 osgi-3=deb https://deb.iteg.at/any any-osgi-3 myapp-2=deb http://deb.mycompany.com/any any-myapp-2 [options] arch=amd64 [osgi-main-packages] osgi-runner-server osgi-runner-client osgi-runner-osgi-bundle-activator osgi-runner-oracle12 osgi-runner-log4j2 osgi-runner-systemd osgi-bundles-common osgi-bundles-jdbc osgi-bundles-remoting osgi-bundles-pdfbox osgi-bundles-dojo osgi-bundles-dmutils osgi-bundles-login-adapter osgi-bundles-login-services osgi-bundles-fancymail-server [myapp-main-packages] myapp-bundles
The packages are downloaded to two subdirectories, osgi-main and myapp-main, respectively.
Start a config file driven download
pba-download -c deb-packages.conf
YUM repositories
All-in-one cmdline
This variant allows to download multiple packages using one or more repositories specified by the -y <repourl> option.
An architecture and a distro must be specified using the option -a <arch> and -d <distro>.
Download RPM package from a YUM repository on the cmdline.
$ pba-download -y http://rpm.clazzes.org/repos/fancyclibs-1/fancyclibs-1.repo -a x86_64 -d 7 fancylog-1 fancylog-1-devel 2018-07-04 22:46:41,236 INFO Loading yum repository from [http://rpm.clazzes.org/repos/fancyclibs-1/fancyclibs-1.repo] 2018-07-04 22:46:41,321 INFO Will try to fetch GPG key [file:///etc/pki/rpm-gpg/RPM-GPG-KEY-pba-builder-iteg-at] from URL [http://rpm.clazzes.org/gpg/RPM-GPG-KEY-pba-builder-iteg-at] 2018-07-04 22:46:41,322 INFO Successfully loaded yum repository with name [fancyclibs-1] and baseurl [http://rpm.clazzes.org/repos/fancyclibs-1/centos$releasever/$basearch/]. 2018-07-04 22:46:41,323 INFO Loading URL [http://rpm.clazzes.org/repos/fancyclibs-1/centos7/x86_64/repodata/repomd.xml]... 2018-07-04 22:46:41,421 INFO Loaded repomd.xml with revision [1492005064] and [6] entries. 2018-07-04 22:46:41,422 INFO Open DB for yum repo [fancyclibs-1] 2018-07-04 22:46:41,422 INFO Loading URL [http://rpm.clazzes.org/repos/fancyclibs-1/centos7/x86_64/repodata/ca6a8999e22d71b589fd72d685a11135eab72318302beb9308d8eb02945f096f-primary.sqlite.bz2]... 2018-07-04 22:46:41,509 INFO Creating sqlite DB under [/tmp/tmpro0l6fgwpbadownload/primary_db] 2018-07-04 22:46:41,510 INFO Found package [fancylog-1] in repository [fancyclibs-1] 2018-07-04 22:46:41,511 INFO Loading URL [http://rpm.clazzes.org/repos/fancyclibs-1/centos7/x86_64/fancylog-1-1.0.3-1.x86_64.rpm]... 2018-07-04 22:46:41,610 INFO Saved file [fancylog-1-1.0.3-1.x86_64.rpm] with checksum [17531db46c7cf186451cc4647e5dbb391813c31e0ddd9caa8d2f53b4f63975f9] 2018-07-04 22:46:41,611 INFO Setting modification time of file [fancylog-1-1.0.3-1.x86_64.rpm] to [2017-04-12 13:51:01] 2018-07-04 22:46:41,612 INFO Found package [fancylog-1-devel] in repository [fancyclibs-1] 2018-07-04 22:46:41,613 INFO Loading URL [http://rpm.clazzes.org/repos/fancyclibs-1/centos7/x86_64/fancylog-1-devel-1.0.3-1.x86_64.rpm]... 2018-07-04 22:46:41,700 INFO Saved file [fancylog-1-devel-1.0.3-1.x86_64.rpm] with checksum [12bd5a9f0ab55ee8e968bcc01c6149b982f4cac56f2473d89f977d0b01e0f91d] 2018-07-04 22:46:41,701 INFO Setting modification time of file [fancylog-1-devel-1.0.3-1.x86_64.rpm] to [2017-04-12 13:51:01]
Download using a config file
You may create a config file, which contains multiple package names and multiple YUM repository URL.
yum-packages.conf config file for large-scale RPM download
[yum] osgi-runner-2=https://rpm.clazzes.org/repos/osgi-runner-2/osgi-runner-2.repo osgi-3=https://rpm.iteg.at/repos/osgi-3/osgi-3.repo rsecm-2=http://rpm.pba.efkon.com/repos/rsecm-2/rsecm-2.repo [options] distro=7 arch=x86_64 [osgi-main-packages] osgi-runner-server osgi-runner-client osgi-runner-osgi-bundle-activator osgi-runner-oracle12 osgi-runner-log4j2 osgi-runner-systemd osgi-bundles-common osgi-bundles-jdbc osgi-bundles-remoting osgi-bundles-pdfbox osgi-bundles-dojo osgi-bundles-dmutils osgi-bundles-login-adapter osgi-bundles-login-services osgi-bundles-fancymail-server [myapp-main-packages] myapp-bundles
The packages are downloaded to two subdirectories, osgi-main and myapp-main, respectively.
Start a config file driven download
pba-download -c yum-packages.conf