How to replace a path in AST with just parsed javascript(string)? - abstract-syntax-tree

https://astexplorer.net/#/gist/70df1bc56b9ee73d19fc949d2ef829ed/7e14217fd8510f0bf83f3372bf08454b7617bce1
I've found now I'm trying to replace an expression and I don't care whats in it.
in this example I've found the this.state.showMenu && this.handleMouseDown portion in
<a
onMouseDown={this.state.showMenu && this.handleMouseDown}
>
I need to convert to:
<a
onMouseDown={this.state.showMenu ? this.handleMouseDown : undefined}
>
how can I do so without explicitly reconstructing the tree? I just want to do something like
path.replaceText("this.state.showMenu ? this.handleMouseDown : undefined")

Here's a transformer that does what you describe:
export default function transformer(file, api) {
const j = api.jscodeshift;
const root = j(file.source)
root
.find(j.JSXExpressionContainer)
.replaceWith(path => {
return j.jsxExpressionContainer(
j.conditionalExpression(
j.identifier(j(path.value.expression.left).toSource()),
j.identifier(j(path.value.expression.right).toSource()),
j.identifier('undefined')
)
)
})
return root.toSource()
}
See it in action here.
You can also just put arbitrary text in the JSXExpressionContainer node:
export default function transformer(file, api) {
const j = api.jscodeshift;
const root = j(file.source)
root
.find(j.JSXExpressionContainer)
.replaceWith(path => {
return j.jsxExpressionContainer(
j.identifier('whatever you want')
)
})
return root.toSource()
}
See this example.
Finally, you don't even need to return a JSXExpressionContainer.
export default function transformer(file, api) {
const j = api.jscodeshift;
const root = j(file.source)
root
.find(j.JSXExpressionContainer)
.replaceWith(path => {
return j.identifier("this isn't valid JS, but works just fine")
})
return root.toSource()
}
See the result here.

You can do this with our DMS Software Reengineering Toolkit.
DMS treats HTML pages as native HTML text with embedded scripting sublanguage, which might be ECMAScript, or VBScript, or something else.
So the process of building a complete HTML "AST" requires that one first
build the pure HTML part, then find all the "onXXXXX" tags and convert them to ASTs in the chosen scripting language. DMS can distinguish AST nodes from different langauges so there's no chance of confusion in understanding the compound AST.
So, first we need to parse the HTML document of interest (code edited for pedagogical reasons):
(local (;; [my_HTML_AST AST:Node]
(includeunique `DMS/Domains/HTML/Component/ParserComponent.par')
);;
(= working_graph (AST:CreateForest))
(= my_HTML_AST (Parser:ParseFile parser working_graph input_file_full_path))
Then we need to walk over the HTML tree, find the JavaScript text fragments, parse them and splice the parsed ECMASCript tree in to replace the text fragment:
(local (;; (includeunique `DMS/Domains/ECMAScript/Components/ParserComponent.par') );;
(ScanNodes my_HTML_AST
(lambda (function boolean AST:Node)
(ifthenelse (!! (~= (AST:GetNodeType ?) GrammarConstants:Rule:Attribute) ; not an HTML attribute
(~= (Strings:Prefix (AST:GetLiteralString (AST:GetFirstChild ?)) `on')) ; not at action attribute
)&&
~t ; scan deeper into tree
(value (local (;; [my_ECMAScript_AST AST:Node]
[ECMASCript_text_stream streams:buffer]
);;
(= ECMAScript_text_stream (InputStream:MakeBufferStream (AST:StringLiteral (AST:GetSecondChild ?))=
(= my_ECMAScript_AST (Parser:ParseStream parser working_graph ECMAScript_text_stream))
(= AST:ReplaceNode ? my_ECMAScript_AST)
(= InputStream:Close my_ECMAScript_text_stream)
~f) ; no need to scan deeper here
)ifthenelse
)lambda
) ; at this point, we have a mixed HTML/ECMAScript tree
)local
If the scripting language can be something else, then this code has to change. If your pages are all HTML + ECMAScript, you can wrap the above stuff into a black box and call it "(ParseHTML)" which is what the other answer assumed happened.
Now for the actual work. OP want to replace a pattern found in his HTML with another. Here DMS shines because you can write those patterns, using the syntax of the targeted language, directly as a DMS Rewrite Rule (see this link for details).
source domain ECMAScript;
target domain ECMAScript;
rule OP_special_rewrite()=expression -> expression
"this.state.showMenu && this.handleMouseDown"
-> "this.state.showMenu ? this.handleMouseDown : undefined "
Now you need to apply this rewrite:
(RSL:Apply my_HTML_AST `OP_special_rewrite') ; applies this rule to every node in AST
; only those that match get modified
And finally regenerate text from the AST:
(PrettyPrinter:PrintStream my_ECMAScript_AST input_file_full_path)
OP's example is pretty simply because he is matching against what amounts to a constant pattern. DMS's rules can be written using all kinds of pattern variables; see above link, and can have arbitrary conditions over the matched pattern and other state information to control whether the rule applies.

Related

Add element to arrays, that are values to a given key name (json transformation with jq)

I'm a jq newbie, and I try to transform a json (a Swagger spec). I want to add an element to the array value of the "parameter" keys:
{
...
"paths": {
"/great/endpoint1": {
"get": {
"parameters": [] <<--- add a value here
}
}
"/great/endpoint2": {
"post": {
"parameters": [] <<-- and here too here too etc.
....
The following jqplay almost works. It adds values to the right arrays, but it has the nasty side effect of also removing the "x-id" value from the root of the input json. It's probably because of a faulty if-condition. As the paths contain a varying string (the endpoint names), I don't know how to write a wildcard path expression to address those, which is why I have tried using walk instead:
https://jqplay.org/s/az56quLZa3
Since the sample data is incomplete, it's difficult to say exactly what you're looking for but it looks like you should be using parameters in the call to walk:
walk(if type=="object" and has("parameters")
then .parameters += [{"extra": "value"}]
else . end)
If you want to restrict the walk to the top-level paths, you would preface the above with: .paths |=

Understanding Elm's Type Signature return types

I am trying to understand elm's type signatures. What does this function return exactly? It appears to be a function that accepts no arguments and returns ...
route : Parser (Page -> a) a
As a learning exercise for myself I'm going to try to answer this. Others will chip in if I get something wrong.
I'm sure you are used to something like
type Person = Adult String | Child String Age
Child is a type that takes two parameters. Parser is the same. But it's definition is pretty formidable
type Parser a b =
Parser (State a -> List (State b))
type alias State value =
{ visited : List String
, unvisited : List String
, params : Dict String String
, value : value
}
That said, you see how Parser is ultimately a wrapper around a function from a State to a list of States. Ultimately it is going to be passed a List of 'unvisited' strings or params; it will progressively 'visit' each one and the result will be combined into the final 'value'.
Next, note that while Parser takes two type parameters - a, b - parseHash is defined
parseHash : Parser (a -> a) a -> Location -> Maybe a
So, your original
route : Parser (Page -> a) a
is going to have to be
route : Parser (Page -> Page) Page
to type check.
To return to your original question, therefore, route is a Parser (which is a very general object) that encapsulates instructions on how to go from one Page to another, and can be used - via parseHash - to tell you what Page to go to next, and that is of course what you would expect from a router.
Hope this gets you started

How to relativize URLs in css files in Hakyll?

In my Hakyll site I have a stylesheet linked into the page:
<link rel="stylesheet" type="text/css" href="/css/my.css">
This CSS contains a #font-face directive linking to a font file:
#font-face {
font-family: "Bla";
src: url("/data/bla.ttf") format("truetype");
}
The problem is that font's URL doesn't get relativized by relativizeUrls even if I move it into a <script> tag inside the page itself. How to solve this problem?
tl;dr – You could use Beerend Lauwers's hakyll-extra package (doesn't seem to be on hackage yet), which provides a relativizeUrl macro. Or, implement your own as follows:
If you don't have too many links, and don't fancy introducing a CSS parser just in order to do this, you could just create a function field - effectively, a macro -- which allows you to call, e.g. relativize("/some/url") from within the page. (I ran into a similar problem, because I wanted to relativize links to a stylesheet for use only by old versions of Internet Explorer; and to TagSoup, the links looked as if they were within comments, so it didn't process them.)
First, we need to write a version of relativizeUrls which just operates on a single URL:
import Data.List as L
-- | Relativize URL. Same logic as "relativizeUrlsWith" in
-- Hakyll.Web.Html.RelativizeUrls, but for just one url.
relativizeUrl :: String -- ^ Path to the site root
-> String -- ^ link to relativize
-> String -- ^ Resulting link
relativizeUrl root = rel
where
isRel :: String -> Bool
isRel x = "/" `L.isPrefixOf` x && not ("//" `L.isPrefixOf` x)
rel x = if isRel x then root ++ x else x
Then, we define a "function field" which can be added to contexts.
import Data.Maybe (maybe)
-- ugh. ugly name.
relativizeFuncField :: Context a
relativizeFuncField = functionField "relativize" relativize
where
relativize :: [String] -> Item a -> Compiler String
relativize args item = do
siteRoot <- getRoot <$> (getRoute $ itemIdentifier item)
arg <- case args of
[arg] -> return arg
_ -> error "relativize: expected only 1 arg"
return $ relativizeUrl siteRoot arg
getRoot :: Maybe String -> String
getRoot = maybe (error "relativize: couldn't get route") toSiteRoot
Then, anywhere you want to use this macro, instead of using, say, defaultContext, use relativizeFuncField <> defaultContext. e.g.:
import Data.Monoid( (<>) )
main =
-- ...
match (fromList ["about.rst", "contact.markdown"]) $ do
route $ setExtension "html"
compile $ pandocCompiler
>>= loadAndApplyTemplate "templates/default.html" (relativizeFuncField <> defaultContext)
>>= relativizeUrls
So, finally, that means that within a file, you can write $relativize("/path/to/file")$ in any spot where TagSoup isn't already relativizing links.
Hope that's of use :)
(This was using Hakyll 4.9.0.0, but I assume other 4.X versions are much the same.)
edited: p.s., many thanks to Beerend Lauwers, who explained Hakyll function fields in his post here
edited again: d'oh. I didn't see that Beerend has actually already put a relativizeUrl function in his hakyll-extra package.
Hakyll's relativizeURLs uses TagSoup to parse and pretty-print HTML, so it can only work on URLs found inside HTML attributes. I don't know of any existing functionality to extend this to CSS rather than just HTML attributes.
The relevant code goes through every tag parsed by TagSoup and applies a function to attributes it recognizes as URLs:
-- | Apply a function to each URL on a webpage
withUrls :: (String -> String) -> String -> String
withUrls f = withTags tag
where
tag (TS.TagOpen s a) = TS.TagOpen s $ map attr a
tag x = x
attr (k, v) = (k, if isUrlAttribute k then f v else v)
(From Hakyll.Web.HTML)
There's no way to change this traversal logic from the provided relativizeURLs compiler so you'll probably have to write your own. Luckily it's pretty simple: it gets the site root (with toSiteRoot), then uses withURLs to apply a function to every URL that turns absolute paths into relative ones.
relativizeUrls item = do
route <- getRoute $ itemIdentifier item
return $ case route of
Nothing -> item
Just r -> fmap (relativizeUrlsWith $ toSiteRoot r) item
relativizeUrlsWith root = withUrls rel
where
isRel x = "/" `isPrefixOf` x && not ("//" `isPrefixOf` x)
rel x = if isRel x then root ++ x else x
(Excerpts from Hakyll.Web.RelativizeURLs).
You'll need to combine this sort of process with a lightweight CSS parser of some sort. It'll look something like this (in pseudocode):
relativizeCssUrls root = renderCSS . fmap relativize . parseCSS
where relativize (URL url)
| isRel url = URL (root <> url)
| otherwise = URL url
relativize other = other
I haven't used any CSS parsing/printing libraries so I can't give you a good suggestion here, but css-text seems like a decent starting point.
I've gone an easier way. I've used this code to obtain root path relative to a current item:
rootPath :: Compiler String
rootPath = (toSiteRoot . fromJust) <$> (getUnderlying >>= getRoute)
And then created a Context with constant field:
fontCtx = do
root <- rootPath
return $ constField "fontRoot" root
Finally, i moved #font-face clause out of a CSS file into HTML one and used my field there:
<style type="text/css">
#font-face {
...
src: url("$fontRoot$/data/bla.ttf") format("truetype");
}
</style>
That context field turned out to be quite useful in other places, like path strings in Javascript code, which I also use.

ANTLR: how to make a rewrite rule to set a lexema as a AST node text

The following grammar rule aims at recognizing expressions such "a-b" in a grammar that generates a AST to evaluate a linear equation:
tokens {
PLUS = '+' ;
MINUS = '-' ;
DIV = '/' ;
EQUAL = '=' ;
MULT = '*' ;
}
minusExpr: (a=multExpr -> $a) (MINUS b=multExpr -> ^(PLUS $a ^(MINUS $b)))*;
The grammar is working correctly. The only problem that I have is that in the output AST, the text of the token is set to "PLUS" instead of "+".
For example, for the equation: x-1=11
it generates the following tree (the grammar has other rules that I haven't copy here):
(= (PLUS x (- 1)) 11)
Instead of the tree:
(= (+ x (- 1)) 11)
I would like to know how to rewrite the rule so that the AST node label is set to "+" instead of "PLUS". Thanks!
The text '+' is just the input that is converted by the lexer to tokens (in that case with a token type of PLUS). You cannot rewrite that as the lexer will always convert your input to tokens (because the parser only works with tokens).
However, each token has the text it was created from internally stored. So when you walk your tree you can get the original text of each token at any time by calling getText() on the CommonToken or the BaseTree class.

XQuery - problem with recursive function

Im new on this project and am going to write, what i thought was a simple thing. A recursive function that writes nested xml elements in x levels (denoted by a variable). So far I have come up with this, but keeps getting a compile error. Please note that i have to generate new xml , not query existing xml:
xquery version "1.0";
declare function local:PrintTest($amount)
{
<test>
{
let $counter := 0
if ($counter <= $amount )
then local:PrintTest($counter)
else return
$counter := $counter +1
}
</test>
};
local:PrintPerson(3)
My error is:
File Untitled1.xquery: XQuery transformation failed
XQuery Execution Error!
Unexpected token - " ($counter <= $amount ) t"
I never understood xquery, and cant quite see why this is not working (is it just me or are there amazingly few resources on the Internet concerning XQuery?)
You have written this function in a procedural manner, XQuery is a functional language.
Each function body can only be a single expression; it looks like you are trying to write statements (which do not exist in XQuery).
Firstly, your let expression must be followed by a return keyword.
return is only used as part of a FLWOR expression, a function always evaluates to a value. As you have written it return is equivalent to /return and so will return a node called return.
The line $counter := $counter + 1 is not valid XQuery at all. You can only set a variable like this with a let expression, and in this case it would create a new variable called counter which replaced the old one, that would be in scope only in the return expression of the variable.
The correct way to do what you are trying to do is to reduce the value of $argument each time the function recurses, and stop when you hit 0.
declare function local:Test($amount)
{
if ($amount == 0)
then ()
else
<test>
{
local:Test($amount - 1)
}
</test>
};
local:Test(3)
Note that I have changed the name of the function to Test. The name "PrintTest" was misleading, as this implies that the function does something (namely, printing). The function in fact just returns a node, it does not do any printing. In a purely functional langauge (which XQuery is quite close to) a function never has any side effects, it merely returns a value (or in this case a node).
The line $counter := $counter + 1 is valid XQuery Scripting.

Resources