I've created my first ever python script and it works fine in the foreground, but when I run it in the background it creates, but fails to write anything to the file. I run the script with command : python -u testit.py &
Please help me understand why this happens.
#!/usr/bin/env python
import time
import datetime
dt = datetime.datetime.now()
dtLog = dt.strftime("ThermoLogs/TempLOG%Y%m%d")
f = open(dtLog,"a")
while True:
dt = datetime.datetime.now()
print('{:%Y-%m-%d %H:%M:%S}'.format(dt))
f.write('{:%Y-%m-%d %H:%M:%S}'.format(dt))
f.write('\n')
time.sleep(5)
The output will arrive in bursts (it is being buffered). With the sleep set to 0.01 I am getting bursts about every 2 seconds. So for a delay of 5 you will get them much less frequent. You may also note that it outputs all pending outputs when you terminate it.
To make the output arrive now call f.flush().
I don't see any difference between foreground and background. However it should not buffer stdout when going to a terminal.
Related
I am attempting to design a front end GUI for a CLI program by the name of eac3to.exe. The problem as I see it is that this program sends all of it's output to a cmd window. This is giving me no end of trouble because I need to get a lot of this output into a GUI window. This sounds easy enough, but I am begining to wonder whether I have found one of AutoIt's limitations?
I can use the Run() function with a windows internal command such as Dir and then get the output into a variable with the AutoIt StdoutRead() function, but I just can't get the output from an external program such as eac3to.exe - it just doesn't seem to work whatever I do! Just for testing purposesI I don't even need to get the output to a a GUI window: just printing it with ConsoleWrite() is good enough as this proves that I was able to read it into a variable. So at this stage that's all I need to do - get the text (usually about 10 lines) that has been output to a cmd window by my external CLI program into a variable. Once I can do this the rest will be a lot easier. This is what I have been trying, but it never works:
Global $iPID = Run("C:\VIDEO_EDITING\eac3to\eac3to.exe","", #SW_SHOW)
Global $ScreenOutput = StdoutRead($iPID)
ConsoleWrite($ScreenOutput & #CRLF)
After running this script all I get from the consolWrite() is a blank line - not the text data that was output as a result of running eac3to.exe (running eac3to without any arguments just lists a screen of help text relating to all the commandline options), and that's what I am trying to get into a variable so that I can put it to use later in the program.
Before I suggest a solution let me just tell you that Autoit has one
of the best help files out there. Use it.
You are missing $STDOUT_CHILD = Provide a handle to the child's STDOUT stream.
Also, you can't just do RUN and immediately call stdoutRead. At what point did you give the app some time to do anything and actually print something back to the console?
You need to either use ProcessWaitClose and read the stream then or, you should read the stream in a loop. Simplest check would be to set a sleep between RUN and READ and see what happens.
#include <AutoItConstants.au3>
Global $iPID = Run("C:\VIDEO_EDITING\eac3to\eac3to.exe","", #SW_SHOW, $STDOUT_CHILD)
; Wait until the process has closed using the PID returned by Run.
ProcessWaitClose($iPID)
; Read the Stdout stream of the PID returned by Run. This can also be done in a while loop. Look at the example for StderrRead.
; If the proccess doesnt end when finished you need to put this inside of a loop.
Local $ScreenOutput = StdoutRead($iPID)
ConsoleWrite($ScreenOutput & #CRLF)
I'm doing long-running experiments in an IPython notebook running on a server, where the typical work cycle is: launch experiment, go to lunch, come back, check progress, check Facebook, check email, check Facebook again, turn off computer, come back, check Facebook, check progress, ...
The problem is that when I close the browser window where the notebook is running, the print/logging outputs are no longer saved in the notebook.
For instance, in my simple experiment:
import time
start_time = time.time()
for i in xrange(5):
print '%s seconds have passed' % (time.time()-start_time)
time.sleep(2)
print 'Done!'
If I run, close the tab, and come back 10 seconds later, I just see whatever the output was when the notebook was last saved. What I expect to see is:
0.000111818313599 seconds have passed
2.00515794754 seconds have passed
4.01105999947 seconds have passed
6.0162498951 seconds have passed
8.01735782623 seconds have passed
Done!
Presumably this will be fixed at some point in the future, but in the mean time is there some easy hack to make it continue printing to notebook output after closing the browser? Bonus points if it works for inline images.
Well, found an ok solution. Solution is in this file:
https://github.com/QUVA-Lab/artemis/blob/master/artemis/fileman/persistent_print.py
With example use:
https://github.com/QUVA-Lab/artemis/blob/master/artemis/fileman/test_persistent_print.py
The demo now looks like:
import time
from general.persistent_print import capture_print, reprint
capture_print()
start_time = time.time()
for i in xrange(5):
print '%s seconds have passed' % (time.time()-start_time)
time.sleep(2)
print 'Done!'
And if I run
reprint()
In the next cell, it will redisplay all the print statements made since capture_print was called. Obviously it would be better if this were unnecessary, but it works for now.
Sometimes I execute a method that takes long to compute
In [1]:
long_running_invocation()
Out[1]:
Often I am interested in knowing how much time it took, so I have to write this:
In[2]:
import time
start = time.time()
long_running_invocation()
end = time.time()
print end - start
Out[2]: 1024
Is there a way to configure my IPython notebook so that it automatically prints the execution time of every call I am making like in the following example?
In [1]:
long_running_invocation()
Out[1] (1.59s):
This ipython extension does what you want: https://github.com/cpcloud/ipython-autotime
load it by putting this at the top of your notebook:
%install_ext https://raw.github.com/cpcloud/ipython-autotime/master/autotime.py
%load_ext autotime
Once loaded, every subsequent cell execution will include the time it took to execute as part of its output.
i haven't found a way to have every cell output the time it takes to execute the code, but instead of what you have, you can use cell magics: %time or %timeit
ipython cell magics
You can now just use the %%time magic at the beginning of the cell like this:
%%time
data = sc.textFile(sample_location)
doc_count = data.count()
doc_objs = data.map(lambda x: json.loads(x))
which when executed will print out an output like:
CPU times: user 372 ms, sys: 48 ms, total: 420 ms
Wall time: 37.7 s
The Simplest way to configure your ipython notebook in a way it automatically shows the execution time without running any %%time or %%timelit or time.time() in each cell, is by using ipython-autotime package.
Install the package in the begining of the notebook
pip install ipython-autotime
and then load the extension by running below
%load_ext autotime
Once you have loaded it, any cell run after this ,will give you the execution time of the cell.
And dont worry if you want to turn it off, just unload the extension by running below
%unload_ext autotime
It is pretty simple and easy to use it whenever you want.
And if you want to check out more, can refer to ipython-autime documentation or its github source
I am trying to pause my code for a little while, time for me to observe the plots.
I tried:
print('A')
something = readline("Press Enter")
print('B')
print('C')
, then there is no pause, the line print('B') is fed to readline and get stored into something and therefore only A and C got printed on the screen. Note that if I add an empty line between Something = readline("Press Enter") and print("B"), then print("B") get printed on the screen but still the console doesn't allow the user to press enter before continuing.
And I tried:
print('A')
Sys.sleep(3)
print('B')
print('C')
The program waits 3 seconds before starting and then run "normally" without doing any pause between print('A') and print('B').
What do I missunderstand?
Here is my R version: R 3.1.1 GUI 1.65 Snow Leopard build (6784)
The problem with readline is that if you paste your script into an R console, or execute it from eg Rstudio, the redline function is read and then the next line of the script is read in as the console entry, which in your case sets the value of something to print('B).
An easy way to get around this is to stick your entire code in a function, then call the function to run it. So, in your case:
myscript = function(){
print('A')
something = readline(prompt = "Press Enter")
print('B')
print('C')
}
myscript()
The output of this for me (in Rstudio, with R version 3.1.1):
[1] "A"
Press Enter
[1] "B"
[1] "C"
This has always felt like a bit of a hack to me, but it's essentially what the readline documentation recommends in its example.
I've never used sleep in my code, so I can't help you there.
Edit to clarify based on comments: This will only work if myscript() is the very last line of your script, or if it is manually entered into the console after running the script to generate the function. Otherwise, you will run into the same problem as before- the next line of code will be automatically entered.
I am enjoying developing inside the ipython notebook, but I am having a problem when I want to write a main() function that reads the command line args (with OptionParser, for example). I want to be able to export the code to a .py file and run it from the command line, but I haven't found a way to have a main() that runs both in the notebook with predefined arguments or from the command line with python and command line args. What is the secret?
In case that is not clear, I would like to do something like this:
if __name__ == '__main__':
# if in the notebook
vals = {'debug':True, 'tag_file': 't.tags'}
options = Object()
for k,v in vals.items():
options.setattr(k,v)
args = 'fname1.txt'
# if running as a command line python script
from optparse import OptionParser
parser = OptionParser()
parser.add_option('-d','--debug',action='store_true',dest='debug')
parser.add_option('-t','--tags',action='store',dest='tag_file')
options,args = parser.parse_args()
You cannot determine that you are in an IPython notebook or a qtconsole, or a simple IPython shell, for the simple reason the 3 can be connected to the same kernel at the same time.
It would be like asking, what color is the current key the user is typing. You could get it by looking the plugged usb devices and look for images on the internet and guess the keyboard color, but nothing guarantees you it will be accurate, nor that it won't change, and user can have multiple keyboard plugged, or even painted keyboard.
It is really the same with the notebook, Even if you determine you are in ZMQKernel, are you speeking to qtconsole or webserver ? Again, you found that you were talking to the webserver, are you talking to JS or Emacs ? And so on and so forth.
The only thing you can do, you can ask the user.
What is reliable, is test wether you are in IPython or not.
If you really but reeaaalllyy want a way, as until now, the notebook is the only thing that can display Javascript. And javascript can execute code in pyton. So you might be able to create something that display JS that send back info to the kernel. And using thread and timer you can say that you were not in a notebook (but you will have a race condition).
Don't worry about the distinction. Just set default values, and unless they are overridden from the command line, use those.
if __name__ == '__main__':
parser = OptionParser()
parser.add_option('-d', '--debug', action='store_true', dest='debug',
default=True)
parser.add_option('-t','--tags',action='store',dest='tag_file',
default='t.tags')
options, args = parser.parse_args()
if not args:
args = ['fname1.txt']