Re: [PATCH] core-count: A new program to count the number of cpu cores (original) (raw)

[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From: Giuseppe Scrivano
Subject: Re: [PATCH] core-count: A new program to count the number of cpu cores
Date: Sun, 01 Nov 2009 03:40:14 +0100
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1.50 (gnu/linux)

Hi Pádraig,

Pádraig Brady <address@hidden> writes:

I do wonder though whether it would be better to have numprocessors() try to return this by default?

num_processors is going to be used by programs as nproc will be used by scripts; all considerations we made for nproc can be applied to num_processors.

I.E. can you think of a use case where someone would want to use the --installed option? BTW "installed" differs from the terms discussed here, and perhaps "online" would be better (if we did want to expose it at all).

I don't see any common use case, except a quick way to see how many processors can potentially be available.

Also I'm wondering why you used the pthread interface to this? I didn't notice pthreadgetaffinitynp() in POSIX for example (is that what the np represents?), so why not call schedgetaffinity directly without needing to link with the pthread library.

Thanks, I am attaching a new version that uses sched instead of pthread. The _np suffix means "non portable".

Also perhaps we should be comparing to /proc/stat just in case /proc/cpuinfo was not showing the online processors: grep '^cpu[0-9]' /proc/stat | wc -l

From what I can see, /proc/cpuinfo shows the number of online processors:

grep '^proc' /proc/cpuinfo | wc -l

2

echo 0 > /sys/devices/system/cpu/cpu1/online

grep '^proc' /proc/cpuinfo | wc -l

1

echo 1 > /sys/devices/system/cpu/cpu1/online

grep '^proc' /proc/cpuinfo | wc -l

2

Regards, Giuseppe

From 31b047ef9f0e83b7f6387bdd7e628cbb17f24079 Mon Sep 17 00:00:00 2001 From: Giuseppe Scrivano <address@hidden> Date: Sat, 31 Oct 2009 18:59:50 +0100 Subject: [PATCH] nproc: A new program to count the number of processors

AUTHORS | 1 + NEWS | 4 + README | 2 +- bootstrap.conf | 2 + doc/coreutils.texi | 39 +++++++++++++ po/POTFILES.in | 1 + src/Makefile.am | 3 + src/nproc.c | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/Makefile.am | 3 + tests/nproc/avail | 31 ++++++++++ tests/nproc/cpuinfo | 33 +++++++++++ tests/nproc/positive | 31 ++++++++++ 12 files changed, 304 insertions(+), 1 deletions(-) create mode 100644 src/nproc.c create mode 100755 tests/nproc/avail create mode 100755 tests/nproc/cpuinfo create mode 100755 tests/nproc/positive

diff --git a/AUTHORS b/AUTHORS index 7095db0..3855622 100644 --- a/AUTHORS +++ b/AUTHORS @@ -51,6 +51,7 @@ mv: Mike Parker, David MacKenzie, Jim Meyering nice: David MacKenzie nl: Scott Bartram, David MacKenzie nohup: Jim Meyering +nproc: Giuseppe Scrivano od: Jim Meyering paste: David M. Ihnat, David MacKenzie pathchk: Paul Eggert, David MacKenzie, Jim Meyering diff --git a/NEWS b/NEWS index 0760775..8155807 100644 --- a/NEWS +++ b/NEWS @@ -60,6 +60,10 @@ GNU coreutils NEWS -- outline -- touch now accepts the option --no-dereference (-h), as a means to change symlink timestamps on platforms with enough support. +** New programs + + nproc: A new program to print the number of processors. + * Noteworthy changes in release 8.0 (2009-10-06) [beta] diff --git a/README b/README index 7545eab..0951b62 100644 --- a/README +++ b/README @@ -11,7 +11,7 @@ The programs that can be built with this package are: csplit cut date dd df dir dircolors dirname du echo env expand expr factor false fmt fold groups head hostid hostname id install join kill link ln logname ls md5sum mkdir mkfifo mknod mktemp mv nice nl nohup - od paste pathchk pinky pr printenv printf ptx pwd readlink rm rmdir + nproc od paste pathchk pinky pr printenv printf ptx pwd readlink rm rmdir runcon seq sha1sum sha224sum sha256sum sha384sum sha512sum shred shuf sleep sort split stat stdbuf stty su sum sync tac tail tee test timeout touch tr true truncate tsort tty uname unexpand uniq unlink uptime users diff --git a/bootstrap.conf b/bootstrap.conf index 4c0f4c7..c273065 100644 --- a/bootstrap.conf +++ b/bootstrap.conf @@ -158,6 +158,7 @@ gnulib_modules=" modechange mountlist mpsort + nproc obstack pathmax perl @@ -190,6 +191,7 @@ gnulib_modules=" save-cwd savedir savewd + sched selinux-at settime sig2str diff --git a/doc/coreutils.texi b/doc/coreutils.texi index ec5bcfb..d703520 100644 --- a/doc/coreutils.texi +++ b/doc/coreutils.texi @@ -84,6 +84,7 @@ * nice: (coreutils)nice invocation. Modify niceness. * nl: (coreutils)nl invocation. Number lines and write files. * nohup: (coreutils)nohup invocation. Immunize to hangups. +* nproc: (coreutils)nproc invocation. Print the number of processors. * od: (coreutils)od invocation. Dump files in octal, etc. * paste: (coreutils)paste invocation. Merge lines of files. * pathchk: (coreutils)pathchk invocation. Check file name portability. @@ -409,6 +410,7 @@ System context * arch invocation:: Print machine hardware name * date invocation:: Print or set system date and time +* nproc invocation:: Print the number of processors * uname invocation:: Print system information * hostname invocation:: Print or set system name * hostid invocation:: Print numeric host identifier @@ -13232,6 +13234,7 @@ information. @menu * date invocation:: Print or set system date and time. * arch invocation:: Print machine hardware name. +* nproc invocation:: Print the number of processors. * uname invocation:: Print system information. * hostname invocation:: Print or set system name. * hostid invocation:: Print numeric host identifier. @@ -13891,6 +13894,42 @@ The program accepts the @ref{Common options} only. @exitstatus address@hidden nproc invocation address@hidden @command{nproc}: Print the number of processors + address@hidden nproc address@hidden Print the number of processors address@hidden system information, printing + address@hidden prints the number of processors. It is not a hardware +inspection tool but a portable way to get how many processes +potentially can be executed in parallel. The result is guaranteed to +be a positive number greater than zero. Synopsis: + address@hidden +nproc address@hidden address@hidden example + +The program accepts the following options. Also see @ref{Common options}. + address@hidden @samp + address@hidden --available address@hidden --available +Print the number of processors available to the current process. It +may be less than the number of installed processors. +If this information is not accessible, then nproc returns the number of +installed processors. By default --available is used. + address@hidden --installed address@hidden --installed +Print the number of installed processors. + address@hidden table + address@hidden + + @node uname invocation @section @command{uname}: Print system information diff --git a/po/POTFILES.in b/po/POTFILES.in index e84e2ea..9a46a9a 100644 --- a/po/POTFILES.in +++ b/po/POTFILES.in @@ -89,6 +89,7 @@ src/mv.c src/nice.c src/nl.c src/nohup.c +src/nproc.c src/od.c src/operand2sig.c src/paste.c diff --git a/src/Makefile.am b/src/Makefile.am index 67c29cc..be1df2b 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -86,6 +86,7 @@ EXTRA_PROGRAMS =
mktemp
mv
nl
+ nproc
nohup
od
paste
@@ -189,6 +190,7 @@ chown_LDADD = $(LDADD) chroot_LDADD = $(LDADD) cksum_LDADD = $(LDADD) comm_LDADD = $(LDADD) +nproc_LDADD = $(LDADD) cp_LDADD = $(LDADD) csplit_LDADD = $(LDADD) cut_LDADD = $(LDADD) @@ -480,6 +482,7 @@ rmdir_SOURCES = rmdir.c prog-fprintf.c uname_SOURCES = uname.c uname-uname.c arch_SOURCES = uname.c uname-arch.c +nproc_SOURCES = nproc.c md5sum_SOURCES = md5sum.c md5sum_CPPFLAGS = -DHASH_ALGO_MD5=1 $(AM_CPPFLAGS) diff --git a/src/nproc.c b/src/nproc.c new file mode 100644 index 0000000..07005ec --- /dev/null +++ b/src/nproc.c @@ -0,0 +1,155 @@ +/* nproc - print the number of processors. + Copyright (C) 2009 Free Software Foundation, Inc. + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see <http://www.gnu.org/licenses/>. / + +/ Written by Giuseppe Scrivano. / + +#include <config.h> +#include <getopt.h> +#include <stdio.h> +#include <sys/types.h> + +#include "system.h" +#include "sched.h" +#include "nproc.h" + +/ The official name of this program (e.g., no g' prefix). */ +#define PROGRAM_NAME "nproc" + +#define AUTHORS proper_name ("Giuseppe Scrivano") + +enum +{ + AVAILABLE_OPTION = CHAR_MAX + 1, + INSTALLED_OPTION +}; + +static struct option const longopts[] = +{ + {"installed", no_argument, NULL, INSTALLED_OPTION}, + {"available", no_argument, NULL, AVAILABLE_OPTION}, + {GETOPT_HELP_OPTION_DECL}, + {GETOPT_VERSION_OPTION_DECL}, + {NULL, 0, NULL, 0} +}; + +void +usage (int status) +{ + if (status != EXIT_SUCCESS) + fprintf (stderr, _("Try %s --help' for more information.\n"), + program_name); + else + { + printf (("Usage: %s [OPTION]...\n"), program_name); + fputs (("
+Print the number of processors.\n
+\n
+"), stdout); + fputs (("
+ --available print the number of processors available to the\n
+ current process\n
+ --installed print the number of installed processors\n
+"), stdout); + + fputs (HELP_OPTION_DESCRIPTION, stdout); + fputs (VERSION_OPTION_DESCRIPTION, stdout); + emit_ancillary_info (); + } + exit (status); +} + +/* Compute the number of available processors. Return 0 on error. */ + +static unsigned long +nproc_available (void) +{ + unsigned long nproc = 0; + +#ifdef CPU_SETSIZE + size_t j; + cpu_set_t cpuset; + CPU_ZERO (&cpuset); + + if (sched_getaffinity (0, sizeof (cpu_set_t), &cpuset)) + return 0; + + for (j = 0; j < CPU_SETSIZE; j++) + if (CPU_ISSET (j, &cpuset)) + nproc++; +#endif + + return nproc; +} + +/* Compute the number of processors. If AVAILABLE returns the number + of processors available to the current process, otherwise the number + of installed processors. In case the number of available processors + can't be computed rollback to the installed processors. The result is + guaranteed to be at least 1. */ + +static unsigned long +nproc (bool available) +{ + if (available) + { + unsigned long available_proc = nproc_available (); + if (available_proc) + return available_proc; + } + + return num_processors (); +} + +int +main (int argc, char **argv) +{ + bool available = true; + initialize_main (&argc, &argv); + set_program_name (argv[0]); + setlocale (LC_ALL, ""); + bindtextdomain (PACKAGE, LOCALEDIR); + textdomain (PACKAGE); + + atexit (close_stdout); + + while (1) + { + int c = getopt_long (argc, argv, "", longopts, NULL); + if (c == -1) + break; + switch (c) + { + case_GETOPT_HELP_CHAR; + + case_GETOPT_VERSION_CHAR (PROGRAM_NAME, AUTHORS); + + case AVAILABLE_OPTION: + available = true; + break; + + case INSTALLED_OPTION: + available = false; + break; + + default: + usage (EXIT_FAILURE); + } + } + + printf ("%lu\n", nproc (available)); + + exit (EXIT_SUCCESS); +} diff --git a/tests/Makefile.am b/tests/Makefile.am index eec31ae..2f89c4e 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -460,6 +460,9 @@ TESTS =
touch/read-only
touch/relative
touch/trailing-slash
+ nproc/avail
+ nproc/cpuinfo
+ nproc/positive
$(root_tests) pr_data =
diff --git a/tests/nproc/avail b/tests/nproc/avail new file mode 100755 index 0000000..3ab04ae --- /dev/null +++ b/tests/nproc/avail @@ -0,0 +1,31 @@ +#!/bin/sh +# Ensure that nproc --installed is greater or equal to nproc --available. + +# Copyright (C) 2009 Free Software Foundation, Inc. + +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +if test "$VERBOSE" = yes; then + set -x + nproc --version +fi + +. $srcdir/test-lib.sh + +available=$(nproc --available) +installed=$(nproc --installed) + +test installed−geinstalled -ge installed−geavailable || fail=1 + +Exit $fail diff --git a/tests/nproc/cpuinfo b/tests/nproc/cpuinfo new file mode 100755 index 0000000..6330c49 --- /dev/null +++ b/tests/nproc/cpuinfo @@ -0,0 +1,33 @@ +#!/bin/sh +# Check that the number of processors reported by nproc is equal to the number +# of processors listed in /proc/cpuinfo, when it is available + +# Copyright (C) 2009 Free Software Foundation, Inc. + +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +if test "$VERBOSE" = yes; then + set -x + nproc --version +fi + +. $srcdir/test-lib.sh + +test -r /proc/cpuinfo || skip_test + +processors=$(nproc --installed) + +test (grep′proc′/proc/cpuinfo∣wc−l)−eq(grep '^proc' /proc/cpuinfo | wc -l) -eq (grep′proc′/proc/cpuinfo∣wc−l)−eqprocessors || fail=1 + +Exit $fail diff --git a/tests/nproc/positive b/tests/nproc/positive new file mode 100755 index 0000000..76efb3f --- /dev/null +++ b/tests/nproc/positive @@ -0,0 +1,31 @@ +#!/bin/sh +# Ensure that nproc returns a non-zero positive number + +# Copyright (C) 2009 Free Software Foundation, Inc. + +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +if test "$VERBOSE" = yes; then + set -x + nproc --version +fi + +. $srcdir/test-lib.sh + +for opt in --available --installed; do + cores=$(nproc $opt) + test $cores -gt 0 || fail=1 +done + +Exit $fail

1.6.5