According to what I have researched there are no illegal characters in the PWD= field of a SQL Server Connection String.
However, using SQL Server Express 2008 I changed the SA password to a GUID, specifically:
{85C86BD7-B15F-4C51-ADDA-3B6A50D89386}
So when connecting via ODBC I use this connection string:
"Driver={SQL Server};Server=.\\MyInstance;Database=Master;UID=SA;PWD={85C86BD7-B15F-4C51-ADDA-3B6A50D89386};"
But it comes back as Login failed for SA.
However, if I change the SA password to something just as long but without {}- it succeeds! Are there certain characters in PWD= that need to be escaped? I tried all different combinations with no luck.
As Microsoft's documentation states (emphasis added) --
Connection strings used by ODBC have the following syntax:
connection-string ::= empty-string[;] | attribute[;] | attribute; connection-string
empty-string ::=
attribute ::= attribute-keyword=[{]attribute-value[}]
attribute-value ::= character-string
attribute-keyword ::= identifier
Attribute values can optionally be enclosed in braces, and it is good practice to do so. This avoids problems when attribute values contain non-alphanumeric characters. The first closing brace in the value is assumed to terminate the value, so values cannot contain closing brace characters.
I would suggest you simply remove the braces when you set the password, and then the connect string you provided above should work fine.
ADDITION
I dug a bit further on Microsoft's site, and found some ABNF rules which may be relevant --
SC = %x3B ; Semicolon
LCB = %x7B ; Left curly brackets
RCB = %x7D ; Right curly brackets
EQ = %x3D ; Equal sign
ESCAPEDRCB = 2RCB ; Double right curly brackets
SpaceStr = *(SP) ; Any number (including 0) spaces
ODBCConnectionString = *(KeyValuePair SC) KeyValuePair [SC]
KeyValuePair = (Key EQ Value / SpaceStr)
Key = SpaceStr KeyName
KeyName = (nonSP-SC-EQ *nonEQ)
Value = (SpaceStr ValueFormat1 SpaceStr) / (ValueContent2)
ValueFormat1 = LCB ValueContent1 RCB
ValueContent1 = *(nonRCB / ESCAPEDRCB)
ValueContent2 = SpaceStr / SpaceStr (nonSP-LCB-SC) *nonSC
nonRCB = %x01-7C / %x7E- FFFF ; not "}"
nonSP-LCB-SC = %x01-1F / %x21-3A / %x3C-7A / %x7C- FFFF ; not space, "{" or ";"
nonSP-SC-EQ = %x01-1F / %x21-3A / %x3C / %x3E- FFFF ; not space, ";" or "="
nonEQ = %x01-3C / %x3E- FFFF ; not "="
nonSC = %x01-003A / %x3C- FFFF ; not ";"
...
ValueFormat1 is recommended to use when there is a need for Value to contain LCB, RCB, or EQ. ValueFormat1 MUST be used when the Value contains SC or starts with LCB.
ValueContent1 MUST be enclosed by LCB and RCB. Spaces before the enclosing LCB and after the enclosing RCB MUST be ignored.
ValueContent1 MUST be contained in ValueFormat1. If there is an RCB in the ValueContent1, it MUST use the two-character sequence ESCAPEDRCB to represent the one-character value RCB.
All of which comes down to... I believe the following connect string should work for you (note that there are 2 left/open braces and 3 right/close braces on the PWD value) --
"Driver={SQL Server};Server=.\\MyInstance;Database=Master;UID=SA;PWD={{85C86BD7-B15F-4C51-ADDA-3B6A50D89386}}};"
According to this page, the only legal "special character" in a name (I think they're talking about the DSN) is the UNDERSCORE:
The ODBC specification (and the SQL specification) states that names
must be in the format of " letter[digit | letter | _]...". The only
special character allowed is an underscore.
There was no reference to "the ODBC Specification". This page says it's the the ODBC 4.0 Spec.
Related
PurchPackingSlipJournalCreate class -> initHeader method have a line;
vendPackingSlipJour.PackingSlipId = purchParmTable.Num;
but i want when i copy and paste ' FDG 2020 ' (all blanks are tab character) in Num area and click okey, write this value as 'FDG2020' in the PackagingSlipId field of the vendPackingSlipJour table.
I tried -> vendPackingSlipJour.PackingSlipId = strRem(purchParmTable.Num, " ");
but doesn't work for tab character.
How can i remove all whitespace characters from string?
Version 1
Try the strAlpha() function.
From the documentation:
Copies only the alphanumeric characters from a string.
Version 2
Because version 1 also deletes allowed hyphens (-), you could use strKeep().
From the documentation:
Builds a string by using only the characters from the first input string that the second input string specifies should be kept.
This will require you to specify all desired characters, a rather long list...
Version 3
Use regular expressions to replace any unwanted characters (defined as "not a wanted character"). This is similar to version 2, but the list of allowed characters can be expressed a lot shorter.
The example below allows alphanumeric characters(a-z,A-Z,0-9), underscores (_) and hyphens (-). The final value for newText is ABC-12_3.
str badCharacters = #"[^a-zA-Z0-9_-]"; // so NOT an allowed character
str newText = System.Text.RegularExpressions.Regex::Replace(' ABC-12_3 ', badCharacters, '');
Version 4
If you know the only unwanted characters are tabs ('\t'), then you can go hunting for those specifically as well.
vendPackingSlipJour.PackingSlipId = strRem(purchParmTable.Num, '\t');
I see some sites, like stackoverflow, use packed cookies where multiple cookies are packed into one. Here's an example:
Set-Cookie: acct=t=&s=; domain=.stackapps.com; expires=Mon, 30-May-2016 20:16:22 GMT; path=/; HttpOnly
Is this just to save sending multiple set-cookie headers, and to avoid sending comma separated cookies on the one set-cookie header? That's allowed--but is it not recommended?
Should the packed cookie just be treated as a single cookie, or does it need to be unpacked and sent back as individual cookies?
I do not know from where the idea of "packed" came about. Those are just cookies with the = sign in the value, or at least should be according to the specs. Let us go through the RFCs and see that:
Set-Cookie: acct=t=&s=; domain=.stackapps.com; expires=...
is exactly the same as
Set-Cookie: acct="t=&s="; domain=.stackapps.com; expires=...
Therefore, it is a single cookie and shall be treated as such.
The answer is rather long, sorry for that. I tried to aim it at people who find the grammar rules found in the RFCs difficult to understand. If you believe that some piece of the grammar is still difficult to understand please point it to me in a comment.
Through the RFCs
The current RFC for the Set-Cookie header is RFC6265, in section 4.1 it has the formal syntax for Set-Cookie:
set-cookie-header = "Set-Cookie:" SP set-cookie-string
set-cookie-string = cookie-pair *( ";" SP cookie-av )
cookie-pair = cookie-name "=" cookie-value
cookie-name = token
cookie-value = *cookie-octet / ( DQUOTE *cookie-octet DQUOTE )
cookie-octet = %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E
; US-ASCII characters excluding CTLs,
; whitespace DQUOTE, comma, semicolon,
; and backslash
token = <token, defined in [RFC2616], Section 2.2>
cookie-av = expires-av / max-age-av / domain-av /
path-av / secure-av / httponly-av /
extension-av
expires-av = "Expires=" sane-cookie-date
sane-cookie-date = <rfc1123-date, defined in [RFC2616], Section 3.3.1>
max-age-av = "Max-Age=" non-zero-digit *DIGIT
; In practice, both expires-av and max-age-av
; are limited to dates representable by the
; user agent.
non-zero-digit = %x31-39
; digits 1 through 9
domain-av = "Domain=" domain-value
domain-value = <subdomain>
; defined in [RFC1034], Section 3.5, as
; enhanced by [RFC1123], Section 2.1
path-av = "Path=" path-value
path-value = <any CHAR except CTLs or ";">
secure-av = "Secure"
httponly-av = "HttpOnly"
extension-av = <any CHAR except CTLs or ";">
That is a little terse but we do not need to got through it all. For a start we have the Set-Cookie: header and a space (SP), then the set-cookie-string which is defined further.
set-cookie-header = "Set-Cookie:" SP set-cookie-string
set-cookie-string is composed of a cookie-pair (defined further), which is the grammar part that interests us, and optionally a set of any number of cookie-av prefixed with ; and a space. The *() construct allows for any number of occurrences (including zero) of the grammar part.
set-cookie-string = cookie-pair *( ";" SP cookie-av )
cookie-av defines the metadata that can be used in the cookie but it is not needed for our proof, therefore we will abandon its discussion.
The cookie-pair on the other hand is a very simple construct: one cookie-name one mandatory = sign and one cookie-value.
cookie-pair = cookie-name "=" cookie-value
The cookie-name is defined as a token which leads us to another RFC, RFC2616. In the section 2.2 of that RFC we find the basic rules that define the token.
cookie-name = token
token = <token, defined in [RFC2616], Section 2.2>
token definition:
CTL = <any US-ASCII control character
(octets 0 - 31) and DEL (127)>
...
token = 1*<any CHAR except CTLs or separators>
separators = "(" | ")" | "<" | ">" | "#"
| "," | ";" | ":" | "\" | <">
| "/" | "[" | "]" | "?" | "="
| "{" | "}" | SP | HT
The 1*<> syntax means any number of occurrences but at least one occurrence. To find the CTLs use man ascii and check the Dec column, SP is space (as we already saw) and HT is the horizontal tab (9 in the ascii table).
The interesting part for us is the fact that a token cannot contain an = character.
Back to RFC6265:
cookie-pair = cookie-name "=" cookie-value
cookie-name stops at the first = character, that first = character is always the = explicit in the grammar. Now, let's finally define the cookie-value
cookie-value = *cookie-octet / ( DQUOTE *cookie-octet DQUOTE )
cookie-octet = %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E
; US-ASCII characters excluding CTLs,
; whitespace DQUOTE, comma, semicolon,
; and backslash
We already saw that the * there means any occurrences including zero (note that empty cookies are allowed by the RFC!). The interesting part the entire cookie-value can be enclosed by double quotes (DQUOTE is the double quote character as you might have guessed).
But the most interesting part is that the = sign (x3D in the ascii) table is allowed as a cookie-octet
/ %x3C-5B / <- right there!
Yet the space (x20) and semicolon (x3B) are disallowed.
Conclusion
Therefore this Set-Cookie header shall be interpreted as
Set-Cookie: acct=t=&s=; domain=.stackapps.com; expires=...
cookie-set-header = "Set-Cookie:" SP set-cookie-string
set-cookie-string = cookie-pair *(";" cookie-av)
cookie-pair = cookie-name "=" cookie-value
cookie-name = "acct"
cookie-value = "t=&s="
And the header sending it back to the server shall be
Cookie: acct=t=&s=
Sending it as follows violates the RFC:
Cookie: acct=t&; s=
I am quite new pyparsing user and have missing match i don't understand:
Here is the text i would like to parse:
polraw="""
set policy id 800 from "Untrust" to "Trust" "IP_10.124.10.6" "MIP(10.0.2.175)" "TCP_1002" permit
set policy id 800
set dst-address "MIP(10.0.2.188)"
set service "TCP_1002-1005"
set log session-init
exit
set policy id 724 from "Trust" to "Untrust" "IP_10.16.14.28" "IP_10.24.10.6" "TCP_1002" permit
set policy id 724
set src-address "IP_10.162.14.38"
set dst-address "IP_10.3.28.38"
set service "TCP_1002-1005"
set log session-init
exit
set policy id 233 name "THE NAME is 527 ;" from "Untrust" to "Trust" "IP_10.24.108.6" "MIP(10.0.2.149)" "TCP_1002" permit
set policy id 233
set service "TCP_1002-1005"
set service "TCP_1006-1008"
set service "TCP_1786"
set log session-init
exit
"""
I setup grammar this way:
KPOL = Suppress(Keyword('set policy id'))
NUM = Regex(r'\d+')
KSVC = Suppress(Keyword('set service'))
KSRC = Suppress(Keyword('set src-address'))
KDST = Suppress(Keyword('set dst-address'))
SVC = dblQuotedString.setParseAction(lambda t: t[0].replace('"',''))
ADDR = dblQuotedString.setParseAction(lambda t: t[0].replace('"',''))
EXIT = Suppress(Keyword('exit'))
EOL = LineEnd().suppress()
P_SVC = KSVC + SVC + EOL
P_SRC = KSRC + ADDR + EOL
P_DST = KDST + ADDR + EOL
x = KPOL + NUM('PId') + EOL + Optional(ZeroOrMore(P_SVC)) + Optional(ZeroOrMore(P_SRC)) + Optional(ZeroOrMore(P_DST))
for z in x.searchString(polraw):
print z
Result set is such as
['800', 'MIP(10.0.2.188)']
['724', 'IP_10.162.14.38', 'IP_10.3.28.38']
['233', 'TCP_1002-1005', 'TCP_1006-1008', 'TCP_1786']
The 800 is missing service tag ???
What's wrong here.
Thanks by advance
Laurent
The problem you are seeing is that in your expression, DST's are only looked for after having skipped over optional SVC's and SRC's. You have a couple of options, I'll go through each so you can get a sense of what all is going on here.
(But first, there is no point in writing "Optional(ZeroOrMore(anything))" - ZeroOrMore already implies Optional, so I'm going to drop the Optional part in any of these choices.)
If you are going to get SVC's, SRC's, and DST's in any order, you could refactor your ZeroOrMore to accept any of the three data types, like this:
x = KPOL + NUM('PId') + EOL + ZeroOrMore(P_SVC|P_SRC|P_DST)
This will allow you to intermix different types of statements, and they will all get collected as part of the ZeroOrMore repetition.
If you want to keep these different types of statements in groups, then you can add a results name to each:
x = KPOL + NUM('PId') + EOL + ZeroOrMore(P_SVC("svc*")|
P_SRC("src*")|
P_DST("dst*"))
Note the trailing '*' on each name - this is equivalent to calling setResultsName with the listAllMatches argument equal to True. As each different expression is matched, the results for the different types will get collected into the "svc", "src", or "dst" results name. Calling z.dump() will list the tokens and the results names and their values, so you can see how this works.
set policy id 233
set service "TCP_1002-1005"
set dst-address "IP_10.3.28.38"
set service "TCP_1006-1008"
set service "TCP_1786"
set log session-init
exit
shows this for z.dump():
['233', 'TCP_1002-1005', 'IP_10.3.28.38', 'TCP_1006-1008', 'TCP_1786']
- PId: 233
- dst: [['IP_10.3.28.38']]
- svc: [['TCP_1002-1005'], ['TCP_1006-1008'], ['TCP_1786']]
If you wrap ungroup on the P_xxx expressions, maybe like this:
P_SVC,P_SRC,P_DST = (ungroup(expr) for expr in (P_SVC,P_SRC,P_DST))
then the output is even cleaner-looking:
['233', 'TCP_1002-1005', 'IP_10.3.28.38', 'TCP_1006-1008', 'TCP_1786']
- PId: 233
- dst: ['IP_10.3.28.38']
- svc: ['TCP_1002-1005', 'TCP_1006-1008', 'TCP_1786']
This is actually looking pretty good, but let me pass on one other option. There are a number of cases where parsers have to look for several sub-expressions in any order. Let's say they are A,B,C, and D. To accept these in any order, you could write something like OneOrMore(A|B|C|D), but this would accept multiple A's, or A, B, and C, but not D. The exhaustive/exhausting combinatorial explosion of (A+B+C+D) | (A+B+D+C) | etc. could be written, or you could maybe automate it with something like
from itertools import permutations
mixNmatch = MatchFirst(And(p) for p in permutations((A,B,C,D),4))
But there is a class in pyparsing called Each that allows to write the same kind of thing:
Each([A,B,C,D])
meaning "must have one each of A, B, C, and D, in any order". And like And, Or, NotAny, etc., there is an operator shortcut too:
A & B & C & D
which means the same thing.
If you want "must have A, B, and C, and optionally D", then write:
A & B & C & Optional(D)
and this will parse with the same kind of behavior, looking for A, B, C, and D, regardless of the incoming order, and whether D is last or mixed in with A, B, and C. You can also use OneOrMore and ZeroOrMore to indicate optional repetition of any of the expressions.
So you could write your expression as:
x = KPOL + NUM('PId') + EOL + (ZeroOrMore(P_SVC) &
ZeroOrMore(P_SRC) &
ZeroOrMore(P_DST))
I looked at using results names with this expression, and the ZeroOrMore's seem to be confusing things, maybe still a bug in how this is done. So you may have to reserve using Each for more basic cases like my A,B,C,D example. But I wanted to make you aware of it.
Some other notes on your parser:
dblQuotedString.setParseAction(lambda t: t[0].replace('"','')) is probably better written
dblQuotedString.setParseAction(removeQuotes). You don't have any embedded quotes in your examples, but it's good to be aware of where your assumptions might not translate to a future application. Here are a couple of ways of removing the defining quotes:
dblQuotedString.setParseAction(lambda t: t[0].replace('"',''))
print dblQuotedString.parseString(r'"This is an embedded quote \" and an ending quote \""')[0]
# prints 'This is an embedded quote \ and an ending quote \'
# removed leading and trailing "s, but also internal ones too, which are
# really part of the quoted string
dblQuotedString.setParseAction(lambda t: t[0].strip('"'))
print dblQuotedString.parseString(r'"This is an embedded quote \" and an ending quote \""')[0]
# prints 'This is an embedded quote \" and an ending quote \'
# removed leading and trailing "s, and leaves the one internal ones but strips off
# the escaped ending quote
dblQuotedString.setParseAction(removeQuotes)
print dblQuotedString.parseString(r'"This is an embedded quote \" and an ending quote \""')[0]
# prints 'This is an embedded quote \" and an ending quote \"'
# just removes leading and trailing " characters, leaves escaped "s in place
KPOL = Suppress(Keyword('set policy id')) is a bit fragile, as it will break if there are any extra spaces between 'set' and 'policy', or between 'policy' and 'id'. I usually define these kind of expressions by first defining all the keywords individually:
SET,POLICY,ID,SERVICE,SRC_ADDRESS,DST_ADDRESS,EXIT = map(Keyword,
"set policy id service src-address dst-address exit".split())
and then define the separate expressions using:
KSVC = Suppress(SET + SERVICE)
KSRC = Suppress(SET + SRC_ADDRESS)
KDST = Suppress(SET + DST_ADDRESS)
Now your parser will cleanly handle extra whitespace (or even comments!) between individual keywords in your expressions.
We are working with Apache Tomcat 7 and trying to setup the Valve Component to store our access logs, ready for processing in SnowPlow.
The problem we have is how to make these logs robust. To give an example - we can separate fields with tabs and extract the user agent string like so:
pattern="%{yyyy-MM-dd}t %{hh:mm:ss}t %{User-Agent}i "
The problem is that the Valve Component does not (as far as I can see) escape %{User-Agent}i, so a stray tab in a useragent will corrupt the data (row will look like it contains four fields, not three).
As far as solutions, unless there's a way of escaping the useragent which I've missed, I can see a couple of solutions:
Use a really obscure field delimiter (or combination of field delimiters) which is very unlikely to crop up in a useragent string. We tried Ctrl-A (HTML ?) but that didn't seem to work
Write a custom AccessLogValve which either supports escaping or sanitizes tabs - perhaps similar to this post Sanitizing Tomcat access log entries
A bit puzzled that I can't find anything else about this online - does nobody parse their Tomcat access logs?
What do you recommend? We're a little stuck...
RFC2616 defines user agent string as
User-Agent = "User-Agent" ":" 1*( product | comment )
Then product is defined as
product = token ["/" product-version]
product-version = token
Following this, tokens are defined as
token = 1*<any CHAR except CTLs or separators>
and separators/CTLs as
separators = "(" | ")" | "<" | ">" | "#"
| "," | ";" | ":" | "\" | <">
| "/" | "[" | "]" | "?" | "="
| "{" | "}" | SP | HT
CTL = <any US-ASCII control character
(octets 0 - 31) and DEL (127)>
We need not to forget comment, which is defined as
comment = "(" *( ctext | quoted-pair | comment ) ")"
ctext = <any TEXT excluding "(" and ")">
quoted-pair = "\" CHAR
CHAR = <any US-ASCII character (octets 0 - 127)>
So if I understand correctly, you should be able to use any separator or CTL as long as you can distinguish comment, which is wrapped in ( and ). If ( appears inside the comment, it should be escaped with \.
In the end, I wrote a custom Tomcat AccessLogValve which:
Introduced a new pattern, 'I', to escape an incoming header
Introduced a new pattern, 'C', to fetch a cookie stored on the response
Re-implemented the pattern 'i' to ensure that "" (empty string) is replaced with "-"
Re-implemented the pattern 'q' to remove the "?" and ensure "" (empty string) is replaced with "-"
Overwrote the 'v' pattern, to write the version of this AccessLogValve, rather than the local server name
It seems to be pretty robust - I haven't had any further issues with unescaped values.
This Wiki article on Base64 URL says
"For this reason, a modified Base64 for URL variant exists, where no padding '=' will be used, and the '+' and '/' characters of standard Base64 are respectively replaced by '-' and '_', so that using URL encoders/decoders is no longer necessary and has no impact on the length of the encoded value, leaving the same encoded form intact for use in relational databases, web forms, and object identifiers in general."
When I try and remove the padding using ASP.NET, I get an error when I get my query strings back. How can I account for the missing padding?
string encoded = GetBase64FromQueryString();
encoded = encoded.PadRight(NextMultiple(encoded.Length, 4), '=');
...
static int NextMultiple(int value, int multiple)
{
int r = value % multiple;
return value + (r != 0 ? multiple - r : 0);
}