What's the best way to run a python code to run as a background service on Linux (CentOS/RHEL) (original) (raw)

Hi all,

I’ve developed a Python-based background service that performs periodic data uploads and file processing. It’s built with multiple modules/packages and runs persistently in the background on CentOS/RHEL systems.

I’m currently managing it with a systemd unit file and using a virtual environment, but I’m exploring more robust, scalable ways to deploy it cleanly across systems.

Question: What is the recommended approach for packaging and deploying a Python service on CentOS/RHEL?

barry-scott (Barry Scott) May 1, 2025, 4:08pm 2

How many servers are you supporting?

You could package it as an rpm and use dnf to install and update it as needed.

Are you dependencies already packaged for Centos?
If so you will not need a venv.

tiran (Christian Heimes) May 1, 2025, 4:22pm 3

You could package your application in an RPM, but that is going to require some effort. You’d have to learn how RPM packaging works and you may have to maintain several RPMs.

A simpler approach is a container + Podman Quadlet. Quadlet lets you run containers with Podman under systemd. Start with ubi9/python-312 container, install your application, push your container to Quay, and then run your container as a Quadlet.

sri12 (sri) May 5, 2025, 5:26am 4

End goal is more than 100 servers

sri12 (sri) May 5, 2025, 5:37am 5

Adding additional info:

I’m using py3.8 and my service should run as root to track some activities.
And, some of our Rhel/centos servers doesn’t support py3.8 by default, it can install upto py3.6 using package managers, so we are manually installing python3.8 using the zip files and setting up the env for testing it.

We would like to deploy it as a bg service which should run always, what’s your recommendation ?

1. If we use containers, we can’t access root level information without providing permissions which requires manual intervention, Is my understanding correct? In future, we are planning to automate the deployment as well
2. If we build rpm using ubuntu (one of the linux servers), is it possible to use it in other OS like RHEL/CENTOS?

barry-scott (Barry Scott) May 5, 2025, 6:40am 6

Have you checked in EPEL for python builds?

Use a fedora/rhel/centos system to build your rpms it the easiest method.
Ubuntu lacks all the needed tooling to build first class rpm packages.

With 100 servers I would definitely use rpms.
I did exactly that at my previous job.
As you noticed containers makes doing root level activities difficult.

You might like to ask questions about packaging over on https://discussion.fedoraproject.org/ where the expertise on Fedora/rhel/centos hangs out and helps people.

Rosuav (Chris Angelico) May 5, 2025, 7:05am 7

I would definitely continue to use systemd. Whatever form of scalability you need, it should be able to incorporate the creation of that unit file, and then systemd will manage all the details.

sri12 (sri) May 5, 2025, 8:13am 8

I’ll check about EPEL,

Follow up question:

• If I build an rpm which must be compatible in both rhel/centos versions, then which is the OS version should I choose to build? lowest OS version?
• Can I use github pipelines to automate this built process?

tiran (Christian Heimes) May 5, 2025, 9:36am 9

It’s up to you. You can run containers with no namespace isolation at all. For example Toolbx containers use the approach to give special containers access to the users home directory, X11/Wayland session, D-Bus, and SSH agent.

sri12 (sri) May 5, 2025, 10:42am 10

Oh, okay, it’s new info to me, I’ll check it

barry-scott (Barry Scott) May 5, 2025, 2:02pm 11

The way that RPMs are built is to use a tool called mock.

You tell mock which OS you are targeting and it uses a build enviroment that is correct for the target.

I use Fedora to build for any of the RHEL/Centos targets.

Maybe I’ve not used them myself.

sri12 (sri) May 6, 2025, 5:25am 12

If I use the RPM build file (which is my Python package), do I still need to set up the Python environment on the target machine? To run the RPM file?

AFAIK, it’s Yes. So, can it be installed and used as a standalone service? That is, I don’t want to perform any Python env setup on my target machine, move the build file, install and run it

barry-scott (Barry Scott) May 6, 2025, 6:56am 13

People with the expertise to tell you how to package for RPM are on https://discussion.fedoraproject.org/
It is where the developers that support python on rhel/centos/fedora are.