On a simple grammar I am in the bad situation that one of my ParseActions is not called.
For me this is strange as parseActions of a base symbol ("logic_oper") and a derived symbol ("cmd_line") are called correctly. Just "pa_logic_cmd" is not called. You can see this on the output which is included at the end of the code.
As there is no exception on parsing the input string, I am assuming that the grammar is (basically) correct.
import io, sys
import pyparsing as pp
def diag(msg, t):
print("%s: %s" % (msg , str(t)) )
def pa_logic_oper(t): diag('logic_oper', t)
def pa_operand(t): diag('operand', t)
def pa_ident(t): diag('ident', t)
def pa_logic_cmd(t): diag('>>>>>> logic_cmd', t)
def pa_cmd_line(t): diag('cmd_line', t)
def make_grammar():
semi = pp.Literal(';')
ident = pp.Word(pp.alphas, pp.alphanums).setParseAction(pa_ident)
operand = (ident).setParseAction(pa_operand)
op_and = pp.Keyword('A')
op_or = pp.Keyword('O')
logic_oper = (( op_and | op_or) + pp.Optional(operand))
logic_oper.setParseAction(pa_logic_oper)
logic_cmd = logic_oper + pp.Suppress(semi)
logic_cmd.setParseAction(pa_logic_cmd)
cmd_line = (logic_cmd)
cmd_line.setParseAction(pa_cmd_line)
grammar = pp.OneOrMore(cmd_line) + pp.StringEnd()
return grammar
if __name__ == "__main__":
inp_str = '''
A param1;
O param2;
A ;
'''
grammar = make_grammar()
print( "pp-version:" + pp.__version__)
parse_res = grammar.parseString( inp_str )
'''USAGE/Output: python test_4.py
pp-version:2.0.3
operand: ['param1']
logic_oper: ['A', 'param1']
cmd_line: ['A', 'param1']
operand: ['param2']
logic_oper: ['O', 'param2']
cmd_line: ['O', 'param2']
logic_oper: ['A']
cmd_line: ['A']
'''
Can anybody give me a hint on this parseAction problem?
Thanks,
The problem is here:
cmd_line = (logic_cmd)
cmd_line.setParseAction(pa_cmd_line)
The first line assigns cmd_line to be the same expression as logic_cmd. You can verify by adding this line:
print("???", cmd_line is logic_cmd)
Then the second line calls setParseAction, which overwrites the parse action of logic_cmd, so the pa_logic_cmd will never get called.
Remove the second line, since you are already testing the calling of the parse action with pa_logic_cmd. You could change to using the addParseAction method instead, but to my mind that is an invalid test (adding 2 parse actions to the same pyparsing expression object).
Or, change the definition of cmd_line to:
cmd_line = pp.Group(logic_cmd)
Now you will have wrapped logic_cmd inside another expression, and you can then independently set and test the running of parse actions on the two different expressions.
Related
The following program is self-documenting. It gives the correct results, but it fails typechecking with mypy --strict.
Maybe there's a bug in mypy. More likely, my type annotations are incorrect. Any ideas?
"""
This code shows a technique for creating user-defined infix operators.
The technique is not my invention. However, I would like to have an implementation that:
a) I understand
b) type-checks with mypy --strict.
This code allows any two-argument function to be called as an operator, using the syntax:
leftArgument <<functionName>> rightArgument
... for the avoidance of doubt, << and >> are literally present in the code.
"""
from typing import Callable, Generic, TypeVar
T = TypeVar("T")
U = TypeVar("U")
V = TypeVar("V")
class WithLeft(Generic[T, U, V]):
def __init__(self, func, left: T) -> None:
self.func = func
self.left = left
def __rshift__(self, right: U) -> V:
return self.func(self.left, right)
class WithRight(Generic[T, U, V]):
def __init__(self, func, right: U) -> None:
self.func = func
self.right = right
def __rlshift__(self, left: T) -> V:
return self.func(left, self.right)
class Op(Generic[T, U, V]):
def __init__(self, func) -> None:
self.func = func
def __rlshift__(self, left: T) -> WithLeft[T, U, V]:
return WithLeft(self.func, left)
def __rshift__(self, right: U) -> WithRight[T, U, V]:
return WithRight(self.func, right)
if __name__ == "__main__":
print("Define the operator <<add>>, which adds two ints.")
#Op
def add(a: int, b: int) -> int:
return a + b
print("Demonstrate that it works.")
print(f"{12 <<add>> 44} correctly prints {12 + 44}")
print('Previous line gives error from mypy:')
print('\tUnsupported operand types for << ("int" and "Op[<nothing>, <nothing>, <nothing>]")')
print()
print("Define the operator <<compose>>, which composes two functions.")
P = TypeVar("P")
Q = TypeVar("Q")
R = TypeVar("R")
#Op
def compose(left: Callable[[Q], P], right: Callable[[R], Q]) -> Callable[[R], P]:
return lambda x: left(right(x))
print("Demonstrate that it works.")
from math import (
cos, # angle in radians, not degrees
log # base e, not base 10
)
logcos: Callable[[float], float] = log <<compose>> cos
print('Previous line gives two errors from mypy:')
print('\tUnsupported operand types for << ("Callable[[SupportsFloat, SupportsFloat], float]" and "Op[<nothing>, <nothing>, <nothing>]")')
print('\tUnsupported operand types for >> ("WithLeft[<nothing>, <nothing>, <nothing>]" and "Callable[[SupportsFloat], float]")')
print()
print(f"{logcos(1)} correctly prints {log(cos(1))}")
Stackoverflow doesn't like this post because it is mostly code.
So I'll repeat the explanation from the code. No need to read this again!
This code shows a technique for creating user-defined infix operators.
The technique is not my invention. However, I would like to have an implementation that:
a) I understand
b) type-checks with mypy --strict.
This code allows any two-argument function to be called as an operator, using the syntax:
leftArgument <> rightArgument
... for the avoidance of doubt, << and >> are literally present in the code.
pyparsing: The below is the code i put up which can parse a nested function call , a logical function call or a hybrid call which nests both the function and a logical function call. The dump() data adds too many unnecessary levels of braces because of grouping. Removing the Group() results in a wrong output. Is there a guideline to use Group(parsers)?
Also the Pyparsing document does'nt detail on how to walk the tree created and not much of data is available out there. Please point me to a link/guide which helps me write the tree walker for recursively parsed data for my test cases.
I will be translating this parsed data to a valid tcl code.
from pyparsing import *
from pyparsing import OneOrMore, Optional, Word, delimitedList, Suppress
# parse action -maker; # from Paul's example
def makeLRlike(numterms):
if numterms is None:
# None operator can only by binary op
initlen = 2
incr = 1
else:
initlen = {0:1,1:2,2:3,3:5}[numterms]
incr = {0:1,1:1,2:2,3:4}[numterms]
# define parse action for this number of terms,
# to convert flat list of tokens into nested list
def pa(s,l,t):
t = t[0]
if len(t) > initlen:
ret = ParseResults(t[:initlen])
i = initlen
while i < len(t):
ret = ParseResults([ret] + t[i:i+incr])
i += incr
return ParseResults([ret])
return pa
line = Forward()
fcall = Forward().setResultsName("fcall")
flogical = Forward()
lparen = Literal("(").suppress()
rparen = Literal(")").suppress()
arg = Word(alphas,alphanums+"_"+"."+"+"+"-"+"*"+"/")
args = delimitedList(arg).setResultsName("arg")
fargs = delimitedList(OneOrMore(flogical) | OneOrMore(fcall) |
OneOrMore(arg))
fname = Word(alphas,alphanums+"_")
fcall << Group(fname.setResultsName('func') + Group(lparen +
Optional(fargs) + rparen).setResultsName('fargs'))
flogic = Keyword("or") | Keyword("and") | Keyword("not")
logicalArg = delimitedList(Group(fcall.setResultsName("fcall")) |
Group(arg.setResultsName("arg")))
#logicalArg.setDebug()
flogical << Group(logicalArg.setResultsName('larg1') +
flogic.setResultsName('flogic') + logicalArg.setResultsName('larg2'))
#logical = operatorPrecedence(flogical, [(not, 1, opAssoc.RIGHT,
makeLRlike(2)),
# (and, 2, opAssoc.LEFT,
makeLRlike(2)),
# (or , 2, opAssoc.LEFT,
makeLRlike(2))])
line = flogical | fcall #change to logical if operatorPrecedence is used
# Works fine
print line.parseString("f(x, y)").dump()
print line.parseString("f(h())").dump()
print line.parseString("a and b").dump()
print line.parseString("f(a and b)").dump()
print line.parseString("f(g(x))").dump()
print line.parseString("f(a and b) or h(b not c)").dump()
print line.parseString("f(g(x), y)").dump()
print line.parseString("g(f1(x), a, b, f2(x,y, k(x,y)))").dump()
print line.parseString("f(a not c) and g(f1(x), a, b, f2(x,y,
k(x,y)))").dump()
#Does'nt work fine yet;
#try changing flogical assignment to logicalArg | flogic
#print line.parseString("a or b or c").dump()
#print line.parseString("f(a or b(x) or c)").dump()
I want to use the argparse library to parse some arguments but I'm struggling to work out what in the myriad of ways you can specify arguments is the simplest way to choose between a few actions. Different actions require different numbers of arguments.
Given the following calls I'd expect the following outputs:
> python MyClass.py action1 foo
Action 1: 12345 - foo
> python MyClass.py action2 20 30
Action 2: 12345 - 20 30
The following seems to work:
import argparse
class MyClass:
def __init__(self, someVar):
self.someVar = someVar
def Action1(self, intToPrint):
print("Print 1: %d - %s"%(self.someVar,intToPrint))
def Action2(self, firstNum, firstString):
print("Print 2: %d - %d %s"%(self.someVar,firstNum, firstString))
def CallAction1(mc, args):
mc.Action1(args.intToPrint)
def CallAction2(mc, args):
mc.Action2(args.firstNum, args.firstString)
def Main():
parser = argparse.ArgumentParser(prog='PythonArgumentParsing.py')
subparsers = parser.add_subparsers(help='commands')
action1Group = subparsers.add_parser('action1', help='action 1 help')
action1Group.add_argument('intToPrint', type=str)
action1Group.set_defaults(func=CallAction1)
action2Group = subparsers.add_parser('action2', help='action 1 help')
action2Group.add_argument('firstNum', type=int)
action2Group.add_argument('firstString', type=str)
action2Group.set_defaults(func=CallAction2)
args = parser.parse_args()
someVar = 12345
mc = MyClass(someVar)
args.func(mc, args)
if __name__ == "__main__":
Main()
...but it seems a little clunky to have to create a CallAction to pass arguments from the parser.
Is there any way to clean this up?
I gather that you are just bothered by needing to write the Call_Action... functions which convert the args namespace into positional parameters for the method calls.
Using keyword parameters might eliminate this need. The following hasn't been tested yet:
def Action1(self, intToPrint=None, **kwargs):
print("Print 1: %d - %s"%(self.someVar,intToPrint))
def Action2(self, firstNum=None, firstString=None, **kwargs):
print("Print 2: %d - %d %s"%(self.someVar,firstNum, firstString))
...
action1Group.set_defaults(func=MyClass.Action1)
...
args.func(mc, **vars(args))
If I've done this right I can pass the whole vars(args) dictionary to the method. It will use the parameters that it needs, and ignore the rest.
argparse makes extensive use of the **kwargs method of passing parameters.
This question already has answers here:
What is the difference between __str__ and __repr__?
(28 answers)
Closed 2 years ago.
I really don't understand where are __str__ and __repr__ used in Python. I mean, I get that __str__ returns the string representation of an object. But why would I need that? In what use case scenario? Also, I read about the usage of __repr__
But what I don't understand is, where would I use them?
__repr__
Called by the repr() built-in function and by string conversions (reverse quotes) to compute the "official" string representation of an object. If at all possible, this should look like a valid Python expression that could be used to recreate an object with the same value (given an appropriate environment).
__str__
Called by the str() built-in function and by the print statement to compute the "informal" string representation of an object.
Use __str__ if you have a class, and you'll want an informative/informal output, whenever you use this object as part of string. E.g. you can define __str__ methods for Django models, which then gets rendered in the Django administration interface. Instead of something like <Model object> you'll get like first and last name of a person, the name and date of an event, etc.
__repr__ and __str__ are similar, in fact sometimes equal (Example from BaseSet class in sets.py from the standard library):
def __repr__(self):
"""Return string representation of a set.
This looks like 'Set([<list of elements>])'.
"""
return self._repr()
# __str__ is the same as __repr__
__str__ = __repr__
The one place where you use them both a lot is in an interactive session. If you print an object then its __str__ method will get called, whereas if you just use an object by itself then its __repr__ is shown:
>>> from decimal import Decimal
>>> a = Decimal(1.25)
>>> print(a)
1.25 <---- this is from __str__
>>> a
Decimal('1.25') <---- this is from __repr__
The __str__ is intended to be as human-readable as possible, whereas the __repr__ should aim to be something that could be used to recreate the object, although it often won't be exactly how it was created, as in this case.
It's also not unusual for both __str__ and __repr__ to return the same value (certainly for built-in types).
Building up and on the previous answers and showing some more examples. If used properly, the difference between str and repr is clear. In short repr should return a string that can be copy-pasted to rebuilt the exact state of the object, whereas str is useful for logging and observing debugging results. Here are some examples to see the different outputs for some known libraries.
Datetime
print repr(datetime.now()) #datetime.datetime(2017, 12, 12, 18, 49, 27, 134411)
print str(datetime.now()) #2017-12-12 18:49:27.134452
The str is good to print into a log file, where as repr can be re-purposed if you want to run it directly or dump it as commands into a file.
x = datetime.datetime(2017, 12, 12, 18, 49, 27, 134411)
Numpy
print repr(np.array([1,2,3,4,5])) #array([1, 2, 3, 4, 5])
print str(np.array([1,2,3,4,5])) #[1 2 3 4 5]
in Numpy the repr is again directly consumable.
Custom Vector3 example
class Vector3(object):
def __init__(self, args):
self.x = args[0]
self.y = args[1]
self.z = args[2]
def __str__(self):
return "x: {0}, y: {1}, z: {2}".format(self.x, self.y, self.z)
def __repr__(self):
return "Vector3([{0},{1},{2}])".format(self.x, self.y, self.z)
In this example, repr returns again a string that can be directly consumed/executed, whereas str is more useful as a debug output.
v = Vector3([1,2,3])
print str(v) #x: 1, y: 2, z: 3
print repr(v) #Vector3([1,2,3])
One thing to keep in mind, if str isn't defined but repr, str will automatically call repr. So, it's always good to at least define repr
Grasshopper, when in doubt go to the mountain and read the Ancient Texts. In them you will find that __repr__() should:
If at all possible, this should look like a valid Python expression that could be used to recreate an object with the same value.
Lets have a class without __str__ function.
class Employee:
def __init__(self, first, last, pay):
self.first = first
self.last = last
self.pay = pay
emp1 = Employee('Ivan', 'Smith', 90000)
print(emp1)
When we print this instance of the class, emp1, this is what we get:
<__main__.Employee object at 0x7ff6fc0a0e48>
This is not very helpful, and certainly this is not what we want printed if we are using it to display (like in html)
So now, the same class, but with __str__ function:
class Employee:
def __init__(self, first, last, pay):
self.first = first
self.last = last
self.pay = pay
def __str__(self):
return(f"The employee {self.first} {self.last} earns {self.pay}.")
# you can edit this and use any attributes of the class
emp2 = Employee('John', 'Williams', 90000)
print(emp2)
Now instead of printing that there is an object, we get what we specified with return of __str__ function:
The employee John Williams earns 90000
str will be informal and readable format whereas repr will give official object representation.
class Complex:
# Constructor
def __init__(self, real, imag):
self.real = real
self.imag = imag
# "official" string representation of an object
def __repr__(self):
return 'Rational(%s, %s)' % (self.real, self.imag)
# "informal" string representation of an object (readable)
def __str__(self):
return '%s + i%s' % (self.real, self.imag)
t = Complex(10, 20)
print (t) # this is usual way we print the object
print (str(t)) # this is str representation of object
print (repr(t)) # this is repr representation of object
Answers :
Rational(10, 20) # usual representation
10 + i20 # str representation
Rational(10, 20) # repr representation
str and repr are both ways to represent. You can use them when you are writing a class.
class Fraction:
def __init__(self, n, d):
self.n = n
self.d = d
def __repr__(self):
return "{}/{}".format(self.n, self.d)
for example when I print a instance of it, it returns things.
print(Fraction(1, 2))
results in
1/2
while
class Fraction:
def __init__(self, n, d):
self.n = n
self.d = d
def __str__(self):
return "{}/{}".format(self.n, self.d)
print(Fraction(1, 2))
also results in
1/2
But what if you write both of them, which one does python use?
class Fraction:
def __init__(self, n, d):
self.n = n
self.d = d
def __str__(self):
return "str"
def __repr__(self):
return "repr"
print(Fraction(None, None))
This results in
str
So python actually uses the str method not the repr method when both are written.
Suppose you have a class and wish to inspect an instance, you see the print doesn't give much useful information
class Person:
def __init__(self, name, age):
self.name = name
self.age = age
p1 = Person("John", 36)
print(p1) # <__main__.Animal object at 0x7f9060250410>
Now see a class with a str, it shows the instance information and with repr you even don't need the print. Nice no?
class Animal:
def __init__(self, color, age, breed):
self.color = color
self.age = age
self.breed = breed
def __str__(self):
return f"{self.color} {self.breed} of age {self.age}"
def __repr__(self):
return f"repr : {self.color} {self.breed} of age {self.age}"
a1 = Animal("Red", 36, "Dog")
a1 # repr : Red Dog of age 36
print(a1) # Red Dog of age 36
I find functools.partial to be extremely useful, but I would like to be able to freeze arguments out of order (the argument you want to freeze is not always the first one) and I'd like to be able to apply it to several methods on a class at once, to make a proxy object that has the same methods as the underlying object except with some of its methods parameters being frozen (think of it as generalizing partial to apply to classes). And I'd prefer to do this without editing the original object, just like partial doesn't change its original function.
I've managed to scrap together a version of functools.partial called 'bind' that lets me specify parameters out of order by passing them by keyword argument. That part works:
>>> def foo(x, y):
... print x, y
...
>>> bar = bind(foo, y=3)
>>> bar(2)
2 3
But my proxy class does not work, and I'm not sure why:
>>> class Foo(object):
... def bar(self, x, y):
... print x, y
...
>>> a = Foo()
>>> b = PureProxy(a, bar=bind(Foo.bar, y=3))
>>> b.bar(2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: bar() takes exactly 3 arguments (2 given)
I'm probably doing this all sorts of wrong because I'm just going by what I've pieced together from random documentation, blogs, and running dir() on all the pieces. Suggestions both on how to make this work and better ways to implement it would be appreciated ;) One detail I'm unsure about is how this should all interact with descriptors. Code follows.
from types import MethodType
class PureProxy(object):
def __init__(self, underlying, **substitutions):
self.underlying = underlying
for name in substitutions:
subst_attr = substitutions[name]
if hasattr(subst_attr, "underlying"):
setattr(self, name, MethodType(subst_attr, self, PureProxy))
def __getattribute__(self, name):
return getattr(object.__getattribute__(self, "underlying"), name)
def bind(f, *args, **kwargs):
""" Lets you freeze arguments of a function be certain values. Unlike
functools.partial, you can freeze arguments by name, which has the bonus
of letting you freeze them out of order. args will be treated just like
partial, but kwargs will properly take into account if you are specifying
a regular argument by name. """
argspec = inspect.getargspec(f)
argdict = copy(kwargs)
if hasattr(f, "im_func"):
f = f.im_func
args_idx = 0
for arg in argspec.args:
if args_idx >= len(args):
break
argdict[arg] = args[args_idx]
args_idx += 1
num_plugged = args_idx
def new_func(*inner_args, **inner_kwargs):
args_idx = 0
for arg in argspec.args[num_plugged:]:
if arg in argdict:
continue
if args_idx >= len(inner_args):
# We can't raise an error here because some remaining arguments
# may have been passed in by keyword.
break
argdict[arg] = inner_args[args_idx]
args_idx += 1
f(**dict(argdict, **inner_kwargs))
new_func.underlying = f
return new_func
Update: In case anyone can benefit, here's the final implementation I went with:
from types import MethodType
class PureProxy(object):
""" Intended usage:
>>> class Foo(object):
... def bar(self, x, y):
... print x, y
...
>>> a = Foo()
>>> b = PureProxy(a, bar=FreezeArgs(y=3))
>>> b.bar(1)
1 3
"""
def __init__(self, underlying, **substitutions):
self.underlying = underlying
for name in substitutions:
subst_attr = substitutions[name]
if isinstance(subst_attr, FreezeArgs):
underlying_func = getattr(underlying, name)
new_method_func = bind(underlying_func, *subst_attr.args, **subst_attr.kwargs)
setattr(self, name, MethodType(new_method_func, self, PureProxy))
def __getattr__(self, name):
return getattr(self.underlying, name)
class FreezeArgs(object):
def __init__(self, *args, **kwargs):
self.args = args
self.kwargs = kwargs
def bind(f, *args, **kwargs):
""" Lets you freeze arguments of a function be certain values. Unlike
functools.partial, you can freeze arguments by name, which has the bonus
of letting you freeze them out of order. args will be treated just like
partial, but kwargs will properly take into account if you are specifying
a regular argument by name. """
argspec = inspect.getargspec(f)
argdict = copy(kwargs)
if hasattr(f, "im_func"):
f = f.im_func
args_idx = 0
for arg in argspec.args:
if args_idx >= len(args):
break
argdict[arg] = args[args_idx]
args_idx += 1
num_plugged = args_idx
def new_func(*inner_args, **inner_kwargs):
args_idx = 0
for arg in argspec.args[num_plugged:]:
if arg in argdict:
continue
if args_idx >= len(inner_args):
# We can't raise an error here because some remaining arguments
# may have been passed in by keyword.
break
argdict[arg] = inner_args[args_idx]
args_idx += 1
f(**dict(argdict, **inner_kwargs))
return new_func
You're "binding too deep": change def __getattribute__(self, name): to def __getattr__(self, name): in class PureProxy. __getattribute__ intercepts every attribute access and so bypasses everything that you've set with setattr(self, name, ... making those setattr bereft of any effect, which obviously's not what you want; __getattr__ is called only for access to attributes not otherwise defined so those setattr calls become "operative" & useful.
In the body of that override, you can and should also change object.__getattribute__(self, "underlying") to self.underlying (since you're not overriding __getattribute__ any more). There are other changes I'd suggest (enumerate in lieu of the low-level logic you're using for counters, etc) but they wouldn't change the semantics.
With the change I suggest, your sample code works (you'll have to keep testing with more subtle cases of course). BTW, the way I debugged this was simply to stick in print statements in the appropriate places (a jurassic=era approach but still my favorite;-).