[FIXED] How do I get the amount of bandwidth used during the execution of a Linux program?

Issue

I need to understand how much bandwidth is used by a program which I execute from a shell on Linux, via subprocess.run in Python.

Maybe a solution already exists, something similar to time?

petur@petur:~$ time foobar
foobar  0.00s user 0.00s system 80% cpu 0.003 total

If not, what libraries would I need create such a utility? (To gather the total incoming and outgoing bandwidth during the execution of a program (including all of its threads/tasks)).

Solution

Here's an idea to get you started, assuming you're talking about network bandwidth. We can trace all writes and reads to sockets using strace. Here's some example output when I run

strace curl example.org
# [...]
recvfrom(5, "HTTP/1.1 200 OK\r\nAge: 550983\r\nCa"..., 102400, 0, NULL, NULL) = 1591
# [...]

This line has information about a single call to the recvfrom system call. Importantly it tells us that the syscall returned 1591, which is the number of bytes read from the socket.

Another example line:

sendto(5, "GET / HTTP/1.1\r\nHost: example.or"..., 74, MSG_NOSIGNAL, NULL, 0) = 74

This tells us that curl wrote 74 bytes to a socket.

So, as a first attempt to estimate the bandwidth used by a program, we can add up all the return values of send* syscalls and all the return values of recv* syscalls.

This isn't perfect - but in practice I suspect it's pretty good. Reasons that it's not perfect:

• there may be other syscalls that can cause network reads/writes
• the syscalls can return errors, which in this case are negative return values
• sockets can be used for local communication too

With that in mind, here's a simple bash function to do the job:

bandwidth() {
  traceout="$(mktemp -u)"
  mkfifo "$traceout"
  strace -o "$traceout" -e recv,recvfrom,recvmsg,send,sendto,sendmsg "$@" &

  awk <"$traceout" '
    BEGIN {rbytes = 0; sbytes = 0}
    /^send/ { sbytes += $NF }
    /^recv/ { rbytes += $NF }
    END {
      print "sent bytes:", sbytes;
      print "received bytes:", rbytes
    }'

  rm "$traceout"
}

Some example usage:

$ bandwidth curl -s -o/dev/null example.org
sent bytes: 178
received bytes: 1599

$ bandwidth curl -s -o/dev/null https://stackoverflow.com
sent bytes: 915
received bytes: 182259

Answered By - stackspace