read python from shell command output

high mark

I understand you might think this is a duplicate question, but so far I haven't found a solution to my own problem.

Currently, I have a C program that constantly produces a stream of data. I want my python program to have a way to read this data so I don't have to do everything in C.

The main feel of a C program is this:

int i = 0;
while(1){
    printf("%d %d\n", i, i+1);
    i++;
}

I read and tried subprocesses in Python and it seems that they all need to wait for the command to complete. (eg this is: keep printing the output of the subprocess while the process is running ) I would like to have some buffering mechanism so that I can record this bitstream and process it line by line. thanks.

A few assumptions:

1) The processed thread can be discarded

2) The buffer/queue size can be no problem since I can control the rate of the source to match the processing speed of the python program.

More background: The C program fundamentally drives the camera's feed (that's why it's written in C), does some OpenCV stuff and outputs int values ​​for the object's position (x,y). The location where the python program needs to perform further processing.

thanks.

John 1024

Here's a way to process the output of a c (or shell) program line by line using the subprocessing module:

from subprocess import Popen, PIPE
# Set up the process
p = Popen("yourprogram", stdout=PIPE, close_fds=True)
count = 0
while True:
    count += 1
    print '%s: %s' % (count, p.stdout.readline().strip())

For demonstration purposes, yourprogramit could be a simple shell script (executable bit set):

#!/bin/sh
while sleep 1s
do
    date
done

Running the python program yields:

$ python test.py
1: Thu Dec  5 23:31:45 PST 2013
2: Thu Dec  5 23:31:46 PST 2013
3: Thu Dec  5 23:31:47 PST 2013
4: Thu Dec  5 23:31:48 PST 2013
5: Thu Dec  5 23:31:49 PST 2013
6: Thu Dec  5 23:31:50 PST 2013
7: Thu Dec  5 23:31:51 PST 2013

It will continue until you terminate it.

Related

read python from shell command output

high mark I understand you might think this is a duplicate question, but so far I haven't found a solution to my own problem. Currently, I have a C program that constantly produces a stream of data. I want my python program to have a way to read this data so I

Shell script: read input from command output

Rahul Why does the following code only give the first line of ps -eaf output in ps.out? while read line; do echo $line>ps.out; done < <(/bin/ps -eaf) Nicola You truncate the file every time, so you only get the last line. You might want to >>instead >.

Shell script: read input from command output

Rahul Why does the following code only give the first line of ps -eaf output in ps.out? while read line; do echo $line>ps.out; done < <(/bin/ps -eaf) Nicola You truncate the file every time, so you only get the last line. You might want to >>instead >.

Run a shell command in python and read the output

Cheetah I have the following: cmd = "ps aux | grep 'java -jar' | grep -v grep | awk '//{print $2}'".split(' ') p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) out, err = p.communicate() print out When I run the command in the console

Run a shell command in python and read the output

Cheetah I have the following: cmd = "ps aux | grep 'java -jar' | grep -v grep | awk '//{print $2}'".split(' ') p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) out, err = p.communicate() print out When I run the command in the console

Run a shell command in python and read the output

Cheetah I have the following: cmd = "ps aux | grep 'java -jar' | grep -v grep | awk '//{print $2}'".split(' ') p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) out, err = p.communicate() print out When I run the command in the console

How to read the output of a shell command

Vivek I'm trying to run a shell script and based on the output determine what node-1 and node-2 are. Currently I am able to get the desired output of the shell command, but not sure how to sort/read it and get the node names. import subprocess cmd = ["olsnodes

Read shell command output in meteor

Mascarpone cheese I am trying to ping a remote server to check if it is online. The process is like this: 1) User inserts target hostname 2) Meteor execute the command 'nmap -p 22 hostname' 3) Meteor reads and parses the output to check the status of the targe

Output of Ruby Shell command differs from shell

jakeonrails When I run this command from terminal: $ grep -rnw 'PageObjects::CompanySettings::InfoPage' spec/**/*_spec.rb | cut -d: -f1 | uniq It returns a result: spec/features/admins/change_payment_method_spec.rb When I run this command (note that the Inf

Output of Ruby Shell command differs from shell

jakeonrails When I run this command from terminal: $ grep -rnw 'PageObjects::CompanySettings::InfoPage' spec/**/*_spec.rb | cut -d: -f1 | uniq It returns a result: spec/features/admins/change_payment_method_spec.rb When I run this command (note that the Inf

Output of Ruby Shell command differs from shell

jakeonrails When I run this command from terminal: $ grep -rnw 'PageObjects::CompanySettings::InfoPage' spec/**/*_spec.rb | cut -d: -f1 | uniq It returns a result: spec/features/admins/change_payment_method_spec.rb When I run this command (note that the Inf

Shell - suppress output from a single command

CDT： I have a rm *.ocommand Makefilethat removes all object files generated during compilation. However, some error messages are output if some .ofiles do not exist . So how to suppress the error message output? Kaibu: In the context of make(more important tha

System R command differs from shell output

Priya I have ubuntu's default python and Anaconda Python installed in my system. When I run the system command in R: > system('which python') /usr/bin/python But from the shell: user@user:~/Documents/Rad/jee/Solver$ which python /home/user/anaconda2/bin/pytho

Shell - suppress output from a single command

CDT： I have a rm *.ocommand Makefilethat removes all object files generated during compilation. However, some error messages are output if some .ofiles do not exist . So how to suppress the error message output? Kaibu: In the context of make(more important tha

How to capture output from bourne shell command

Rajat Gupta I want to execute a bourne shell command, capture its stdout and its stderr (separately), and its exit code. User 6654769 You can use subprocess (part of stdlib): import subprocess cmd = input("enter your command: " ) result = subprocess.run(cmd,

System R command differs from shell output

Priya I have ubuntu's default python and Anaconda Python installed in my system. When I run the system command in R: > system('which python') /usr/bin/python But from the shell: user@user:~/Documents/Rad/jee/Solver$ which python /home/user/anaconda2/bin/pytho

Shell - suppress output from a single command

CDT： I have a rm *.ocommand Makefilethat removes all object files generated during compilation. However, some error messages are output if some .ofiles do not exist . So how to suppress the error message output? Kaibu: In the context of make(more important tha

Remove output from find command - Shell

cynic 77 I have a shell script that is installing a version of Tomcat on my system. I can print the found version stdoutusing that findcommand . However, I also get output for directories and files not found. How can I remove it from the output and only show t

System R command differs from shell output

Priya I have ubuntu's default python and Anaconda Python installed in my system. When I run the system command in R: > system('which python') /usr/bin/python But from the shell: user@user:~/Documents/Rad/jee/Solver$ which python /home/user/anaconda2/bin/pytho

System R command differs from shell output

Priya I have ubuntu's default python and Anaconda Python installed in my system. When I run the system command in R: > system('which python') /usr/bin/python But from the shell: user@user:~/Documents/Rad/jee/Solver$ which python /home/user/anaconda2/bin/pytho

System R command differs from shell output

Priya I have ubuntu's default python and Anaconda Python installed in my system. When I run the system command in R: > system('which python') /usr/bin/python But from the shell: user@user:~/Documents/Rad/jee/Solver$ which python /home/user/anaconda2/bin/pytho

System R command differs from shell output

Priya I have ubuntu's default python and Anaconda Python installed in my system. When I run the system command in R: > system('which python') /usr/bin/python But from the shell: user@user:~/Documents/Rad/jee/Solver$ which python /home/user/anaconda2/bin/pytho

Using grep in Python command output to shell script

Unlawful I've been struggling with this simple example for the past two hours: I have this line: python -c 'import yum, pprint; yb = yum.YumBase(); pprint.pprint(yb.conf.yumvar, width=1)' this makes: Loaded plugins: product-id {'arch': 'ia32e', 'basearch': '

python save output of shell command to text file

BoJack Knight I would like to save the output of a shell command to a text file via Python. Here is my actual, very basic python code: EDIT Here is the final script, thanks for your help :) import subprocess ip_adress_4 = 0 pr = open("pointer_record.txt",

Get the output of a shell command in Python (cron)?

Exceen I'm trying to write a script that detects my current wifi connection and starts a socks proxy on my local machine. The script itself works fine when I execute it myself. Since I want everything to be automated, I thought about using crontab. The only pr

Get the output of a shell command in Python (cron)?

Exceen I'm trying to write a script that detects my current wifi connection and starts a socks proxy on my local machine. The script itself works fine when I execute it myself. Since I want everything to be automated, I thought about using crontab. The only pr

Is there any way to read lines from command output?

Marcus Thornton I have a preprocessing command to output a file ./preprocess.sh > preprocessed_file and preprocessed_filewill be used like this while read line do ./research.sh $line & done < preprocessed_file rm -f preprocessed_file Is there any wa

Is there any way to read lines from command output?

Marcus Thornton I have a preprocessing command to output a file ./preprocess.sh > preprocessed_file and preprocessed_filewill be used like this while read line do ./research.sh $line & done < preprocessed_file rm -f preprocessed_file Is there any wa