Authors Sarah Julia Kriesch
License CC-BY-NC-SA
How to manage more than 14,000 Software
Packages for multiple Linux Distributions
openSUSE Release Engineering
Sarah Julia Kriesch
openSUSE, Accenture, FAU
sarah.kriesch@opensuse.org
AdaLovelace on LiberaChat
@sjkriesch
www.opensuse.org
Agenda
●
My way to become an openSUSE Release Engineer
●
openSUSE and the openSUSE Project
●
Release Management
●
The mainframe architecture s390x
●
Packaging with OBS
●
Relationship between openSUSE Leap and SUSE Linux Enterprise
●
Security issues within a community project
●
Automated tests with openQA
●
openSUSE & IBM
●
The Open Mainframe Project
●
The Linux Distributions Working Group
●
Q&A
My way becoming an openSUSE Release
Engineer
●
Computer Science Expert – System Integration
●
First tasks in wiki translations and software
(IHK) translations
●
Linux Systems Administrator ●
Creation of bug reports
●
Systems Engineer ●
Attending conferences (booths, presentations)
●
Bachelor Studies in Computer Science,
TH Nürnberg
●
Global Coordinator Translations
●
Lead AG Open Source ●
Founder of the Heroes Team (Infrastructure)
●
Bachelor Thesis at IBM ●
2 years openSUSE Board Member
●
Senior Consultant at Accenture (part-time) ●
Support of the Release Engineering Team
●
Focus on Mainframes, Multi Cloud/Platform and
Kubernetes ●
Teamlead openSUSE zSystems
●
Master Computer Science, FAU (part-time) ●
openSUSE Release Engineer for s390x
The openSUSE Project
●
Community Open Source Project
●
Multiple Linux distributions for
different goals
●
Many sub projects (OBS, openQA,
Yast)
●
More than 400 Contributors
https://www.opensuse.org/
Our main Linux distributions
●
Stable release with yearly releases
●
Rolling Release
●
Closed to daily releases
●
Latest release version 15.5
●
Foundation for development of other
●
Same foundation as SUSE Linux
distributions (containerization, minimal
Enterprise systems, …)
●
Useful for servers and Linux ●
Used also for container images
Beginners
●
Target Group Developers
Roles at openSUSE
●
openSUSE Board – conflict management & decision
making
●
openSUSE Release Engineering – technical decisions &
announcements
●
Release Managers (additional Program Manager tasks)
●
Translators
●
Package Maintainers - updates from upstream
●
Upstream Developers
●
QA Engineers – automated tests
Package Maintainer
Package Maintainer
Test Test
Stages Dev
Dev Stages Contributor
Factory Release
Manager
x86 arm arm64 s390x ppcle riscv
Release
Engineers
The Release Engineering Team
●
Responsible for successful releases
●
Managing/forwarding bug reports to Bugowners
●
Fixing architecture related issues
Publishing release announcements and managing websites
●
for releases
Technical decisions (together with SUSE)
●
Collecting feature requests
●
Agile status update meeting in a weekly basis
●
Representatives for x86, s390x, arm, Power
●
Mainframes
●
Large high-performance
computer systems
●
Big Endian systems
●
Architecture s390x
●
Used for mission-critical data
●
Thousands of VMs can run on
such a system https://www.ibm.com/it-infrastructure/z/hardware
Open Build Service
●
Open Source Software for building iso images, packages (rpm,
deb) and container images
●
Integrated CI/CD pipelines (green success, red failed, red
unresolvable, broken)
●
Supported Linux distributions:
– openSUSE/SLES
– Fedora/RHEL
– Debian, Ubuntu
– Scientific Linux
– CentOS Stream, Mageia
– AppImage, Univention, openEuler, Raspbian, ArchLinux
https://openbuildservice.org
Structure in the Open Build Service
●
development project with different software
●
dependency checks automated before factory
●
manual checks for critical software
●
forwarding to architecture specific ports
●
building of packages and iso (some container) images on build hardware in a
data center
●
Automated container releases for container registries possible
●
Publishing/Releasing after successful tests with openQA
Example package:
https://build.opensuse.org/package/show/Java:Factory/java-17-openjdk
Container registry: https://registry.opensuse.org
How do you handle software
dependencies in AMOS?
Software Bill of Materials in Spec Files
●
Software dependencies (BuildRequires, Requires,
Provides) in Spec files
Example: BuildRequires: gcc-c++ >= 7
●
Definition of special version numbers possible
●
Software will use only this specific software versions
then
●
Reason: Otherwise Release Managers would have to
update the SBOM daily
Command Line for Release Engineers and
Package Maintainers
●
Command osc available for most Linux distributions
●
Building packages on your own device or sending to
OBS
Package dependencies in the terminal:
osc whatdependson openSUSE:Factory llvm16 standard x86_64
https://openbuildservice.org/help/manuals/obs-user-guide/
cha.obs.osc.html
How to manage versions in different
Linux distributions
●
If-else cases for Linux distributions with their different
requirements
●
Programming languages are maintained for different versions
as an example
●
Example: MozillaFirefox
%if 0%{?sle_version} >= 150000 && 0%{?sle_version} <= 150500
BuildRequires: python39
BuildRequires: python39-curses
BuildRequires: python39-devel
%else
BuildRequires: python3 >= 3.7
BuildRequires: python3-curses
BuildRequires: python3-devel
%endif
openSUSE Leap and SUSE Linux Enterprise
https://www.suse.com/c/how-suse-builds-its-
enterprise-linux-distribution-part-5/
Security issues in open source communities
●
Upstream projects have got nothing more than
their default CI/CD pipelines
●
Security scan environments can be expensive
●
Enterprise companies (and other people) should
report, that security issues can be fixed!
●
openSUSE in relationship to SUSE Security
CVEs in openSUSE
I have found a SUSE has got
security issue. Can I enough CVE
receive a CVE numbers. What has
number, please? happened?
I have fixed already That is CVE-2022-1271.
something with zgrep Send me more details,
in AppArmor. I want to that I can create a
reference it. database entry.
security@suse.de
openQA
●
Automated tests after the build in OBS
●
Developed tool by SUSE QA Engineers
●
Used by openSUSE, SUSE, Fedora, Debian,
Gentoo Linux, AlmaLinux, Rocky Linux (public)
●
Automotive software companies are using it also
http://open.qa/
Automated tests with openQA
https://openqa.opensuse.org/group_overview/34
Our Reviews and tests
https://en.opensuse.org/openSUSE:Factory_development_model
openSUSE & IBM
●
Officially supported Linux distributions:
- SLES, RHEL, Ubuntu
●
Unofficial support for community Linux distributions
●
Fixes for Linux kernel, compiler, frameworks, network
●
Bugreports forwarded via Linux Program Manager at IBM
●
Benefit of knowledge about the Linux on Z department @ IBM
●
Development infrastructure via the LinuxONE OSS Community
Cloud
Development Collaboration with other
Linux Distributions?
LinuxONE (OSS) Community Cloud
●
VMs on LinuxONE sponsored by IBM
●
Free access (120 days) for single open-source
contributors (SLES, RHEL, Ubuntu)
●
Long-term access for open-source projects
available
●
https://linuxone.cloud.marist.edu/#/login
The Open Mainframe Project
●
Founded 2015
●
Focal point for deployment and
usage of Linux and Open Source
in a mainframe computing
environment
●
Project under the Linux
Foundation
Working Groups
Linux Distributions Working Group
Structure
Founders/ (Co-)Chairs:
– Sarah Julia Kriesch (openSUSE, Accenture)
– Elizabeth K. Joseph (IBM)
●
One Representative for every Linux distribution
(required for input)
●
Sponsor: SUSE
Our Goals
●
Create a place to collaborate across Linux
Distributions via an OMP mailing list, wiki, and chat
●
Provide a space for distributions to request for help
on their port
●
Ensure any and all infrastructure required is
available for supporting the ports
●
Better support from IBM to fix s390x specific bugs
https://wiki.openmainframeproject.org/display/LinuxDistrosWG/
Linux+Distributions+Working+Group
Our collaborative process
●
Problem discussions on mailing lists
●
Reproducing issues (sometimes)
Open discussions on the mailing list
●
Forwarding issues/ideas of improvement for
IBM
●
Monthly meetings (come together) with
review
Collaboration as a benefit
●
Upstream contributions available for all
●
Lowering research & development costs
(at IBM and in the community)
●
Same solutions for all Linux distributions
●
Sharing knowledge between communities
●
Increasing innovation (diverse community ideas)
●
Accelerating Linux development for s390x
You can achieve more together than alone!
Reasons for such required forwarding
●
Many open source projects everywhere
●
IBM does not know all Linux integrated software
(especially new ones)
●
IBM Maintainers are only for base projects (strategic open
source projects) available
●
Linux Distribution Maintainers know their requirements
●
Connection between IBM and upstream projects is
missing
Conclusion
●
You can not keep 14,000 software packages up to date
alone!
●
Most of the release process has to be automated
●
Use available tools for receiving the overview
●
Trust to the community/communities
●
Collaborate inside and outside/upstream
Recommended books
●
Wade, Karsten: The Open Source Way
●
Cotton, Ben: Program Management for
Open Source Projects
●
Brock, Amanda: Open Source Law, Policy
and Practice
●
Mozilla: Open Leadership Training Series
(https://mozilla.github.io/open-leadership-
training-series/)
Q&A
creative commons licensed (BY-NC-SA)