How can I set FileSizeLimit validation for flask_wtf.file.FileField?
We need to add a custom validator like this:
from wtforms.validators import ValidationError
def FileSizeLimit(max_size_in_mb):
max_bytes = max_size_in_mb*1024*1024
def file_length_check(form, field):
if len(field.data.read()) > max_bytes:
raise ValidationError(f"File size must be less than {max_size_in_mb}MB")
field.data.seek(0)
return file_length_check
Then pass the validator to the file uploader field like this:
uploaded_file = FileField('Upload your file', [FileRequired(), FileSizeLimit(max_size_in_mb=2)])
Credits:
Thanks to #yomajo for pointing out how to reset the filestream pointer after measuring the filesize.
The above answer works to validate, but will fail when actually saving the file, due to exhausted file stream in size validation at len(field.data.read())
To save someone's hours, what I actually ended up with:
def FileSizeLimit(max_size_in_mb):
max_bytes = max_size_in_mb*1024*1024
def file_length_check(form, field):
if len(field.data.read()) > max_bytes:
raise ValidationError(
f'File size is too large. Max allowed: {max_size_in_mb} MB')
field.data.seek(0)
return file_length_check
The easiest method is to use app.config['MAX_CONTENT_LENGTH']
app.config['MAX_CONTENT_LENGTH'] = limit_in_mb * 1024 * 1024
Related
I'm trying to isolate, from a bam file, those sequencing reads that have insertions longer than number (let's say 50bp). I guess I can do that using the cigar but I don't know any easy way to parse it and keep only the reads that I want. This is what I need:
Read1 -> 2M1I89M53I2M
Read2 -> 2M1I144M
I should keep only Read1.
Thanks!
Most likely I'm late, but ...
Probably you want MC tag, not CIGAR. I use BWA, and information on insertions is stored in the MC tag. But I may mistake.
Use pysam module to parse BAM, and regular expressions to parse MC tags.
Example code:
import pysam
import re
input_file = pysam.AlignmentFile('input.bam', 'rb')
output_file = pysam.AlignmentFile('found.bam', 'wb', template = input_file)
for Read in input_file:
try:
TagMC = Read.get_tag('MC')
except KeyError:
continue
InsertionsTags = re.findall('\d+I', TagMC)
if not InsertionsTags: continue
InsertionLengths = [int(Item[:-1]) for Item in InsertionsTags]
MinLength = min(InsertionLengths)
if MinLength > 50: output_file.write(Read)
input_file.close()
output_file.close()
Hope that helps.
Using the built-in "Import Data..." functionality we can import a properly formatted text file (like CSV and/or tab-delimited) as an image. It is rather straight forward to write a script to do so. However, my scripting approach is not efficient - which requires me to loop through each raw (use the "StreamReadTextLine" function) so it takes a while to get a 512x512 image imported.
Is there a better way or an "undocumented" script function that I can tap in?
DigitalMicrograph offers an import functionality via the File/Import Data... menu entry, which will give you this dialog:
The functionality evoked by this dialog can also be accessed by script commands, with the command
BasicImage ImageImportTextData( String img_name, ScriptObject stream, Number data_type_enum, ScriptObject img_size, Boolean lines_are_rows, Boolean size_by_counting )
As with the dialog, one has to pre-specify a few things.
The data type of the image.
This is a number. You can find out which number belongs to which image data type by, f.e., creating an image outputting its data type:
image img := Realimage( "", 4, 100 )
Result("\n" + img.ImageGetDataType() )
The file stream object
This object describes where the data is stored. The F1 help-documention explains how one creates a file-stream from an existing file, but essentially you need to specify a path to the file, then open the file for reading (which gives you a handle), and then using the fileHandle to create the stream object.
string path = "C:\\test.txt"
number fRef = OpenFileForReading( path )
object fStream = NewStreamFromFileReference( fRef, 1 )
The image size object
This is a specific script object you need to allocate. It wraps image size information. In case of auto-detecting the size from the text, you don't need to specify the actual size, but you still need the object.
object imgSizeObj = Alloc("ImageData_ImageDataSize")
imgSizeObj.SetNumDimensions(2) // Not needed for counting!
imgSizeObj.SetDimensionSize(0,10) // Not used for counting
imgSizeObj.SetDimensionSize(1,10) // Not used for counting
Boolean checks
Like with the checkboxes in the UI, you spefic two conditions:
Lines are Rows
Get Size By Counting
Note, that the "counting" flag is only used if "Lines are Rows" is also true. Same as with the dialog.
The following script improrts a text file with couting:
image ImportTextByCounting( string path, number DataType )
{
number fRef = OpenFileForReading( path )
object fStream = NewStreamFromFileReference( fRef, 1 )
number bLinesAreRows = 1
number bSizeByCount = 1
bSizeByCount *= bLinesAreRows // Only valid together!
object imgSizeObj = Alloc("ImageData_ImageDataSize")
image img := ImageImportTextData( "Imag Name ", fStream, DataType, imgSizeObj, bLinesAreRows, bSizeByCount )
return img
}
string path = "C:\\test.txt"
number kREAL4_DATA = 2
image img := ImportTextByCounting( path, kREAL4_DATA )
img.ShowImage()
I'm trying to write a validator for a Django-CMS FilerImageField. The following validator function is used for a default ImageField. When I copy it to the new Model, it will crash with the message 'int' object has no attribute 'file'. Obviously a different type of value is being passed to the validator function. I can't seem to find information on what kind of data is being passed to the validator. How do I correctly reference the file so I can get_image_dimensions()?
def validate(fieldfile_obj):
width, height = get_image_dimensions(fieldfile_obj.file) #crash
if width > 1000:
raise ValidationError("This is wrong!")
Ok I found it. fieldfile_obj in this case contains the primary key of a Image record. The solution was to get an instance of filer.models.Image and pass this instance's file property to the validator function:
from filer.models import Image
# ... code ...
def validate(fieldfile_obj):
image = Image.objects.get(pk=fieldfile_obj)
width, height = get_image_dimensions(image.file)
if width > 1000:
raise ValidationError("This is wrong!")
I'm trying to upload a photo to vk.com using QtWebKit module. The problem I'm facing is inability to properly fill input(type="file")'s value. Here's some related code I use:
def upload():
print 'uploading...'
photoInput = web.page().mainFrame().documentElement().findFirst('input[id="photos_upload_input"]')
assert photoInput, 'No input found'
photoInput.setAttribute('value', '/Users/elmigranto/Downloads/stuff.png')
print photoInput.evaluateJavaScript('return this.value;').toString()
It's reasonable to note, that filling value of file input is impossible from Javascript due to browser security policy. However, it should be possible using Qt API, more specifically, QWebElement::setAttribute() method. And that's what I did… with no effect (well, photoInput.attribute('value') returns expected result, but photoInput.evaluateJavaScript('return this.value;').toString() returns empty string, input's onchange handler is also not triggered).
Setting other attributes is no problem, for example, QWebElement::addClass() works like a charm.
Any help would be super great.Thanks.
The setAttribute method might still not work for security reasons.
But you can redefine the function QWebPage::chooseFile that should normally open the upload dialog and return the filename so that it returns a static file name without opening the dialog, and activate that upload by simulating a "return" key press on the input element.
This seems to work:
from PyQt4.QtCore import *
from PyQt4.QtGui import *
from PyQt4.QtWebKit import *
import sys
class WebPage(QWebPage):
def __init__(self, parent = None):
super(WebPage, self).__init__(parent)
self.overrideUpload = None
def chooseFile(self, originatingFrame, oldFile):
if self.overrideUpload is None:
return super(WebPage, self).chooseFile(originatingFrame, oldFile)
result = self.overrideUpload
self.overrideUpload = None
return result
def setUploadFile(self, selector, filename):
button = self.mainFrame().documentElement().findFirst(selector)
self.overrideUpload = filename
# set the focus on the input element
button.setFocus();
# and simulate a keypress event to make it call our chooseFile method
webview.event(QKeyEvent(QEvent.KeyPress, Qt.Key_Enter, Qt.NoModifier))
def upload():
print 'uploading...'
page.setUploadFile('input[id="photos_upload_input"]',
'/Users/elmigranto/Downloads/stuff.png')
# The change seems to be asynchronous, at it isn't visible
# just after the previous call
app = QApplication(sys.argv)
webview = QWebView()
page = WebPage(webview)
webview.setPage(page)
source = '''
<form action="#">
Select a file: <input type="file" id="photos_upload_input">
<input type="submit">
</form>
'''
webview.loadFinished.connect(upload)
webview.show()
webview.setHtml(source)
sys.exit(app.exec_())
I am using qt's undo framework , which use qundocommand to do some application support undo.
Is there an easy way I can use to save those qundocommand to a file and reload it?
There's no built-in way. I don't think it's very common to save the undo stack between sessions. You'll have to serialize the commands yourself by iterating through the commands on the stack, and saving each one's unique data using QDataStream. It might look something like this:
...
dataStream << undoStack->count(); // store number of commands
for (int i = 0; i < undoStack->count(); i++)
{
// store each command's unique information
dataStream << undoStack->command(i)->someMemberVariable;
}
...
Then you would use QDataStream again to deserialize the data back into QUndoCommands.
You can use QFile to handle the file management.
Use Qt's serialization as described here:
Serialization with Qt
Then within your QUndoCommands you can use a temp file to write the data to it:
http://qt-project.org/doc/qt-4.8/qtemporaryfile.html
However this might cause you an issue since each file is kept open and so on some platforms (Linux) you may run out of open file handles.
To combat this you'd have to create some other factory type object which handles your commands - then this could pass in a reference to a QTemporaryFile automatically. This factory/QUndoCommand care taker object must have the same life time as the QUndoCommands. If not then the temp file will be removed from disk and your QUndoCommands will break.
The other thing you can do is use QUndoCommand as a proxy to your real undo command - this means you can save quite a bit of memory since when your undo command is saved to file you can delete the internal pointer/set it to null. Then restore it later.
Here's a PyQt solution for serializing/pickling QUndoCommands. The tricky part was getting the parent to call __init__ first, then the children. This method relies on all the children's __setstate__ to be called before the parent's, which happens upon pickling as children are returned in the parent's __getstate__.
class UndoCommand(QUndoCommand):
"""
For pickling
"""
def __init__(self, text, parent=None):
QUndoCommand.__init__(self, text, parent)
self.__parent = parent
self.__initialized = True
# defined and initialized in __setstate__
# self.__child_states = {}
def __getstate__(self):
return {
**{k: v for k, v in self.__dict__.items()},
'_UndoCommand__initialized': False,
'_UndoCommand__text': self.text(),
'_UndoCommand__children':
[self.child(i) for i in range(self.childCount())]
}
def __setstate__(self, state):
if hasattr(self, '_UndoCommand__initialized') and \
self.__initialized:
return
text = state['_UndoCommand__text']
parent = state['_UndoCommand__parent'] # type: UndoCommand
if parent is not None and \
(not hasattr(parent, '_UndoCommand__initialized') or
not parent.__initialized):
# will be initialized in parent's __setstate__
if not hasattr(parent, '_UndoCommand__child_states'):
setattr(parent, '_UndoCommand__child_states', {})
parent.__child_states[self] = state
return
# init must be called on unpickle-time to recreate Qt object
UndoCommand.__init__(self, text, parent)
for child in state['_UndoCommand__children']:
child.__setstate__(self.__child_states[child])
self.__dict__ = {k: v for k, v in state.items()}
#staticmethod
def from_QUndoCommand(qc: QUndoCommand, parent=None):
if type(qc) == QUndoCommand:
qc.__class__ = UndoCommand
qc.__initialized = True
qc.__parent = parent
children = [qc.child(i) for i in range(qc.childCount())]
for child in children:
UndoCommand.from_QUndoCommand(child, parent=qc)
return qc