From this question
How do I use a (generic) vector in go?
I tried to create a new vector but the compiler says it is undefined:
$ 6g -V
6g version release.r60.3 9516
$ cat > vectest.go <<.
> package main
>
> import vector "container/vector"
> import "fmt"
>
> func main() {
> vec := vector.New(0);
> buf := make([]byte,10);
> vec.Push(buf);
>
> for i := 0; i < vec.Len(); i++ {
> el := vec.At(i).([]byte);
> fmt.Print(el,"\n");
> }
> }
> .
$ 6g vectest.go
vectest.go:7: undefined: vector.New
What might be wrong ?
weekly.2011-10-18
The container/vector package has been deleted. Slices are better.
SliceTricks: How to do vector-esque things with slices.
I revised your convertToLCD code to have better performance: 5,745 ns/op versus 19,003 ns/op.
package main
import (
"fmt"
"strconv"
)
const (
lcdNumerals = `
_ _ _ _ _ _ _ _
| | | _| _||_||_ |_ ||_||_|
|_| ||_ _| | _||_| ||_| _|
`
lcdWidth = 3
lcdHeight = 3
lcdLineLen = (len(lcdNumerals) - 1) / lcdWidth
)
func convertToLCD(n int) string {
digits := strconv.Itoa(n)
displayLineLen := len(digits)*lcdWidth + 1
display := make([]byte, displayLineLen*lcdHeight)
for i, digit := range digits {
iPos := i * lcdWidth
digitPos := int(digit-'0') * lcdWidth
for line := 0; line < lcdHeight; line++ {
numeralPos := 1 + lcdLineLen*line + digitPos
numeralLine := lcdNumerals[numeralPos : numeralPos+lcdWidth]
displayPos := displayLineLen*line + iPos
displayLine := display[displayPos : displayPos+lcdWidth]
copy(displayLine, string(numeralLine))
if i == len(digits)-1 {
display[displayLineLen*(line+1)-1] = '\n'
}
}
}
return string(display)
}
func main() {
fmt.Printf("%s\n", convertToLCD(1234567890))
}
Output:
_ _ _ _ _ _ _ _
| _| _||_||_ |_ ||_||_|| |
||_ _| | _||_| ||_| _||_|
It's true there is no vector.New in r60.3, but rather than patch up this code, you should learn the new append function. It made the vector package unnecessary, and in fact the package was removed some time ago from the weekly releases.
Related
I'm trying to load a sqlite file in memory.
I use "VACUUM INTO" like here, but my in-memory database remains empty.
I don't know what's wrong. :
diskDbPath := `file:C:\test.db`
memDbPath := "file:memdb?mode=memory&cache=shared"
diskDb, _ := sql.Open("sqlite3", diskDbPath)
rowcount := 0
diskDb.QueryRow("SELECT count(*) FROM sqlite_master WHERE type = 'table'").Scan(&rowcount)
fmt.Println("disk db:", rowcount, "tables")
memDb, _ := sql.Open("sqlite3", memDbPath)
_, err := diskDb.Exec("VACUUM INTO '" + memDbPath + "'; ")
if err != nil {
panic(err)
}
memDb.QueryRow("SELECT count(*) FROM sqlite_master WHERE type = 'table'").Scan(&rowcount)
fmt.Println("mem db:", rowcount, "tables")
/*
output :
disk db: 9 tables
mem db: 0 tables
*/
driver : github.com/mattn/go-sqlite3 v1.14.7, go version : go1.16.2 windows/amd64
A few tries later...
It seems to work if you ping the memory base before the vacuum.
I guess the in-memory db is not really initialized without a first contact.
diskDbPath := `file:C:\test.db`
memDbPath := "file:memdb?mode=memory&cache=shared"
diskDb, _ := sql.Open("sqlite3", diskDbPath)
rowcount := 0
diskDb.QueryRow("SELECT count(*) FROM sqlite_master WHERE type = 'table'").Scan(&rowcount)
fmt.Println("disk db:", rowcount, "tables")
memDb, _ := sql.Open("sqlite3", memDbPath)
memDb.Ping() // <
_, err := diskDb.Exec("VACUUM INTO '" + memDbPath + "'; ")
if err != nil {
panic(err)
}
memDb.QueryRow("SELECT count(*) FROM sqlite_master WHERE type = 'table'").Scan(&rowcount)
fmt.Println("mem db:", rowcount, "tables")
/*
output :
disk db: 9 tables
mem db: 9 tables
*/
Summary: Dagster run configurations for Dagit vs. PyTest appear to be incompatible for my project
I've been getting errors trying to run pytest on a pipeline and I'd really appreciate any pointers. I've consistently gotten errors of the form:
dagster.core.errors.DagsterInvalidConfigError:
Error in config for pipeline ephemeral_write_myfunc_to_redis_solid_pipeline
Error 1: Undefined field "myfunc_df_to_list" at path root:solids.
Expected: "{ myfunc_list?: { outputs?: [{ result?: { json: { path: String } pickle: { path: String } } }] }
write_myfunc_to_redis?:..."
A few notes about the project:
dagster, version 0.9.15
my pipeline runs in Dagit without errors for the same configuration
the unit tests run for the individual solids that comprise the pipeline
Failed solutions: I've tried populating the configuration files with solids that define the outputs as each pytest error has recommended, but they all have led to errors more opaque than the one before it.
My solids are:
#solid(required_resource_keys={"db"})
def get_myfunc_df(context, query: String) -> myfuncDF:
do something
return myfuncDF
#solid
def myfunc_df_to_list(context, df: myfuncDF) -> List:
do something
return List
#solid(required_resource_keys={"redis"})
def write_myfunc_to_redis(context, myfunc_list:List) -> None:
write to redis return None
And my pipeline is a chain of these solids
#pipeline(
mode_defs=filter_modes(MODES),
preset_defs=filter_presets(PRESETS),
tags={"type": "myproject"},
)
def myfunc_to_redis_pipeline():
df = get_myfunc_df()
myfunc_list = myfunc_df_to_list(df)
write_myfunc_to_redis(myfunc_list)
My test code in test_main.py is
#pytest.mark.myfunc
def test_myfunc_to_redis_pipeline(self):
res = execute_pipeline(myfunc_to_redis_pipeline,
preset="test",)
assert res.success
assert len(res.solid_result_list) == 4
for solid_res in res.solid_result_list:
assert solid_res.success
Where the preset "test" is defined with the run configuration in a yaml file:
resources:
db:
config:
file_path: test.csv
^ This is where it's throwing the most errors and I've been iterating through different permutations of solids to add ala:
solids:
get_myfunc_df:
inputs:
query:
value: select 1
but it hasn't solved the problem yet. Is there any reason the Solids for a test would need their output defined despite the fact that while running in Dagit, only the input solid needs to have a definition?
Is this error indicative of something else being amiss?
edit: Here is the stack trace from tox --verbose
self = <repos.myfunc.myfunc.dagster.tests.test_main.Test_myfunc testMethod=test_myfunc_df>
#pytest.mark.myfunc
def test_myfunc_df(self):
"""myfunc"""
result = execute_solid(
get_myfunc_df,
mode_def=test_mode,
run_config=run_config,
> input_values={"query": "SELECT 1"},
)
repos/myfunc/myfunc/dagster/tests/test_main.py:29:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.tox/repo-myfunc/lib/python3.7/site-packages/dagster/utils/test/__init__.py:324: in execute_solid
raise_on_error=raise_on_error,
.tox/repo-myfunc/lib/python3.7/site-packages/dagster/core/execution/api.py:335: in execute_pipeline
raise_on_error=raise_on_error,
.tox/repo-myfunc/lib/python3.7/site-packages/dagster/core/telemetry.py:90: in wrap
result = f(*args, **kwargs)
.tox/repo-myfunc/lib/python3.7/site-packages/dagster/core/execution/api.py:375: in _logged_execute_pipeline
tags=tags,
.tox/repo-myfunc/lib/python3.7/site-packages/dagster/core/instance/__init__.py:586: in create_run_for_pipeline
pipeline_def, run_config=run_config, mode=mode,
.tox/repo-myfunc/lib/python3.7/site-packages/dagster/core/execution/api.py:644: in create_execution_plan
environment_config = EnvironmentConfig.build(pipeline_def, run_config, mode=mode)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pipeline_def = <dagster.core.definitions.pipeline.PipelineDefinition object at 0x1359f6210>
run_config = {'resources': {'ge_data_context': {'config': {'ge_root_dir': '/Users/this_user/Workspace/drizly-dagster/repos/datas...cause_you_bought/dagster/tests/test.csv'}}}, 'solids': {'get_myfunc_df': {'inputs': {'query': {'value': 'select 1'}}}}}
mode = 'test'
#staticmethod
def build(pipeline_def, run_config=None, mode=None):
"""This method validates a given run config against the pipeline config schema. If
successful, we instantiate an EnvironmentConfig object.
In case the run_config is invalid, this method raises a DagsterInvalidConfigError
"""
from dagster.config.validate import process_config
from dagster.core.definitions.executor import ExecutorDefinition
from dagster.core.definitions.intermediate_storage import IntermediateStorageDefinition
from dagster.core.definitions.system_storage import SystemStorageDefinition
from .composite_descent import composite_descent
check.inst_param(pipeline_def, "pipeline_def", PipelineDefinition)
run_config = check.opt_dict_param(run_config, "run_config")
check.opt_str_param(mode, "mode")
mode = mode or pipeline_def.get_default_mode_name()
environment_type = create_environment_type(pipeline_def, mode)
config_evr = process_config(environment_type, run_config)
if not config_evr.success:
raise DagsterInvalidConfigError(
"Error in config for pipeline {}".format(pipeline_def.name),
config_evr.errors,
> run_config,
)
E dagster.core.errors.DagsterInvalidConfigError: Error in config for pipeline ephemeral_get_myfunc_df_solid_pipeline
E Error 1: Undefined field "inputs" at path root:solids:get_myfunc_df. Expected: "{ outputs?: [{ result?: { csv: { path: (String | { env: String }) sep?: (String | { env: String }) } parquet: { path: (String | { env: String }) } pickle: { path: (String | { env: String }) } table: { path: (String | { env: String }) } } }] }".
.tox/repo-myfunc/lib/python3.7/site-packages/dagster/core/system_config/objects.py:101: DagsterInvalidConfigError
_______________________________________________________________________ Test_myfunc.test_write_myfunc_to_redis ________________________________________________________________________
self = <repos.myfunc.myfunc.dagster.tests.test_main.Test_myfunc testMethod=test_write_myfunc_to_redis>
#pytest.mark.myfunc
def test_write_myfunc_to_redis(self):
"""Test redis write"""
records = [
("k", "v"),
("k2", "v2"),
]
result = execute_solid(
write_myfunc_to_redis,
mode_def=test_mode,
input_values={"myfunc_list": records},
> run_config=run_config,
)
repos/myfunc/myfunc/dagster/tests/test_main.py:56:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.tox/repo-myfunc/lib/python3.7/site-packages/dagster/utils/test/__init__.py:324: in execute_solid
raise_on_error=raise_on_error,
.tox/repo-myfunc/lib/python3.7/site-packages/dagster/core/execution/api.py:335: in execute_pipeline
raise_on_error=raise_on_error,
.tox/repo-myfunc/lib/python3.7/site-packages/dagster/core/telemetry.py:90: in wrap
result = f(*args, **kwargs)
.tox/repo-myfunc/lib/python3.7/site-packages/dagster/core/execution/api.py:375: in _logged_execute_pipeline
tags=tags,
.tox/repo-myfunc/lib/python3.7/site-packages/dagster/core/instance/__init__.py:586: in create_run_for_pipeline
pipeline_def, run_config=run_config, mode=mode,
.tox/repo-myfunc/lib/python3.7/site-packages/dagster/core/execution/api.py:644: in create_execution_plan
environment_config = EnvironmentConfig.build(pipeline_def, run_config, mode=mode)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pipeline_def = <dagster.core.definitions.pipeline.PipelineDefinition object at 0x135d39490>
run_config = {'resources': {'ge_data_context': {'config': {'ge_root_dir': '/Users/this_user/Workspace/drizly-dagster/repos/datas...cause_you_bought/dagster/tests/test.csv'}}}, 'solids': {'get_myfunc_df': {'inputs': {'query': {'value': 'select 1'}}}}}
mode = 'test'
#staticmethod
def build(pipeline_def, run_config=None, mode=None):
"""This method validates a given run config against the pipeline config schema. If
successful, we instantiate an EnvironmentConfig object.
In case the run_config is invalid, this method raises a DagsterInvalidConfigError
"""
from dagster.config.validate import process_config
from dagster.core.definitions.executor import ExecutorDefinition
from dagster.core.definitions.intermediate_storage import IntermediateStorageDefinition
from dagster.core.definitions.system_storage import SystemStorageDefinition
from .composite_descent import composite_descent
check.inst_param(pipeline_def, "pipeline_def", PipelineDefinition)
run_config = check.opt_dict_param(run_config, "run_config")
check.opt_str_param(mode, "mode")
mode = mode or pipeline_def.get_default_mode_name()
environment_type = create_environment_type(pipeline_def, mode)
config_evr = process_config(environment_type, run_config)
if not config_evr.success:
raise DagsterInvalidConfigError(
"Error in config for pipeline {}".format(pipeline_def.name),
config_evr.errors,
> run_config,
)
E dagster.core.errors.DagsterInvalidConfigError: Error in config for pipeline ephemeral_write_myfunc_to_redis_solid_pipeline
E Error 1: Undefined field "get_myfunc_df" at path root:solids. Expected: "{ myfunc_list?: { outputs?: [{ result?: { json: { path: String } pickle: { path: String } } }] } write_myfunc_to_redis?: { outputs?: [{ result?: { json: { path: String } pickle: { path: String } } }] } }".
.tox/repo-myfunc/lib/python3.7/site-packages/dagster/core/system_config/objects.py:101: DagsterInvalidConfigError
=============================================================================== short test summary info ===============================================================================
FAILED repos/myfunc/myfunc/dagster/tests/test_main.py::Test_myfunc::test_myfunc_df - dagster.core.errors.DagsterInvalidConfigError: Error in config for pipeli...
FAILED repos/myfunc/myfunc/dagster/tests/test_main.py::Test_myfunc::test_write_myfunc_to_redis - dagster.core.errors.DagsterInvalidConfigError: Error in conf
Solution below works
The key issue was that the pipeline required solids to be defined in the config as written and the solids were being passed both that same config and input_values in their test function. My change was to remove "input_values" as an argument and pass them via the run configuration. Since my interstitial solids require more complex objects and my configuration file is yaml, I made the following addition to all of my solid tests:
this_solid_run_config = copy.deepcopy(run_config)
input_dict = {"df": pd.DataFrame(['1', '2'], columns = ['key', 'value'])}
this_solid_run_config.update({"solids":
{"myfunc_df_to_list":
{"inputs":input_dict
}
}
}
)
Based on the stack trace, the failure is coming from this:
result = execute_solid(
get_myfunc_df,
mode_def=test_mode,
run_config=run_config,
input_values={"query": "SELECT 1"},
)
The solid input "query" should be passed from either "input_values" param or "run_config" param but not both. Happy to keep digging if that doesn't resolve your issue.
I would like to forward the Actix-Web request body to the response body (something like echo) but it gives a mismatched types error.
use actix_web::*;
use futures::future::ok;
use futures::Future;
fn show_request(
request: &actix_web::HttpRequest
) -> Box<Future<Item=HttpResponse, Error=Error>> {
request
.body()
.from_err::<actix_web::error::PayloadError>()
.map(move |f| {
Box::new(ok(actix_web::HttpResponse::Ok()
.content_type("text/plain")
.body(f)))
})
}
pub fn index(scope: actix_web::Scope<()>) -> actix_web::Scope<()> {
scope.handler("", |req: &actix_web::HttpRequest| {
show_request(req)
})
}
fn main() {
actix_web::server::new(|| {
vec![
actix_web::App::new()
.scope("", index)
.boxed(),
]
}).bind("127.0.0.1:8000")
.expect("Can not bind to port 8000")
.run();
}
[package]
name = "temp"
version = "0.1.0"
authors = ["John"]
edition = "2018"
[dependencies]
actix-web = "0.7"
futures = "0.1"
error:
error[E0308]: mismatched types
--> src/proj.rs:50:2
|
49 | ) -> Box<Future<Item=HttpResponse, Error=Error>> {
| ------------------------------------------- expected `std::boxed::Box<(dyn futures::Future<Error=actix_web::Error, Item=actix_web::HttpResponse> + 'static)>` because of return type
50 | request
| _____^
51 | | .body()
52 | | .from_err::<actix_web::error::PayloadError>()
53 | | .map(move |f| {
... |
56 | | .body(f)))
57 | | })
| |__________^ expected struct `std::boxed::Box`, found struct `futures::Map`
|
= note: expected type `std::boxed::Box<(dyn futures::Future<Error=actix_web::Error, Item=actix_web::HttpResponse> + 'static)>`
found type `futures::Map<futures::future::FromErr<actix_web::dev::MessageBody<actix_web::HttpRequest>, actix_web::error::PayloadError>, [closure#src/.rs:53:8: 57:4]>`
Why does this error occur and how can I fix it?
You are trying to return a Future without Boxing, you are Boxing the response in Map's closure, not the expected Future. Using futures::future::ok is not necessary because your request.body is already future.
fn show_request(
request: &actix_web::HttpRequest,
) -> Box<Future<Item = HttpResponse, Error = Error>> {
Box::new(request.body().map_err(|e| e.into()).map(move |f| {
actix_web::HttpResponse::Ok()
.content_type("text/plain")
.body(f)
}))
}
I want to get my leetcode ranking, But I know about html and JavaScript just a little. After a lot of try, I get this output.
aQuaYi's ranking is Ranking: {[{ pc.ranking }]}
source is
package main
import (
"fmt"
"log"
"github.com/PuerkitoBio/goquery"
)
func showRanking(username string) {
URL := fmt.Sprintf("https://leetcode.com/%s", username)
doc, err := goquery.NewDocument(URL)
if err != nil {
log.Fatal(err)
}
ranking, _ := doc.Find("div.panel-body").Find("span.ranking").Attr("data-content")
fmt.Printf("%s's ranking is %v", username, ranking)
}
func main() {
showRanking("aQuaYi")
}
Please help me finish this code, Thank you very much.
func getRanking(username string) string {
URL := fmt.Sprintf("https://leetcode.com/%s/", username)
fmt.Println(URL)
data := getRaw(URL) // or your way to get raw html page down
str := string(data)
i := strings.Index(str, "ng-init")
j := i + strings.Index(str[i:], "ng-cloak")
str = str[i:j]
i = strings.Index(str, "(")
j = strings.Index(str, ")")
str = str[i:j]
strs := strings.Split(str, ",")
ans := strs[5]
i = strings.Index(ans, "'")
j = 2 + strings.Index(ans[2:], "'")
return ans[i+1 : j]
}
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I am working on a tool that generates HTML from an XML description. I am looking for a library in scala that can help me generate CSS styles for the html elements.
Not sure I understand exactly what you are looking for, but I know http://liftweb.net/ has many tools for dealing with xml, html, and css. You can use the lift libraries without using the full framework.
you are looking for something like this : https://github.com/axiak/scala-css-parser
Run it like that
java -jar CSSRewriter.jar sources/input.css -t where/this/will/go
https://github.com/axiak/scala-css-parser/blob/master/src/SimpleCSSParser.scala
import annotation.tailrec
import collection.mutable.ArrayBuffer
import util.parsing.combinator._
import java.io.File
// See http://www.w3.org/TR/css3-syntax/#grammar0
class SimpleCSSParser extends JavaTokenParsers {
protected override val whiteSpace = """(\s|//.*|(?m)/\*(\*(?!/)|[^*])+\*/)+""".r
// Lexical symbols
def h = "[0-9a-fA-F]".r
def nonascii = "[\200-\377]"
def unicode = "\\[0-9a-fA-F]{1,6}".r
def escape = unicode | "\\[ -~\200-\377]".r
def nmstart = "[a-zA-Z]" | nonascii | escape
def nmchar = "[a-zA-Z0-9-]" | nonascii | escape
override def stringLiteral = ("\""+"""([^"\p{Cntrl}\\]|\\[\\/bfnrt"]|\\u[a-fA-F0-9]{4})*"""+"\"").r | ("\'"+"""([^'\p{Cntrl}\\]|\\[\\/bfnrt']|\\u[a-fA-F0-9]{4})*"""+"\'").r
override def ident = """[*#_]?-?[a-zA-Z_][a-zA-Z0-9_-]*""".r
def name = rep1(nmchar)
def CDO = "<!--"
def CDC = "-->"
def INCLUDES = "~="
def DASHMATCH = "|="
def url = rep("[!#$%&*-~".r | nonascii | escape)
def IMPORT_SYM = "(?i)#import".r
def PAGE_SYM = "(?i)#page".r
def MEDIA_SYM = "(?i)#media".r
def FONT_FACE_SYM = "(?i)#font-face".r
def CHARSET_SYM = "(?i)#charset".r
def NAMESPACE_SYM = "(?i)#namespace".r
def IMPORTANT_SYM = "!important" | ("!" ~ "important")
def EMS = decimalNumber ~ "em"
def EXS = decimalNumber ~ "ex"
def RESOLUTION = decimalNumber ~ "(?i)dpi".r
def LENGTH = decimalNumber ~ "(?i)(?:px|cm|mm|in|pt|pc)".r
def ANGLE = decimalNumber ~ "(?i)(?:deg|rad|grad)".r
def TIME = decimalNumber ~ "(?i)(?:s|ms)".r
def FREQ = decimalNumber ~ "(?i)(?:Hz|kHz)".r
def DIMEN = decimalNumber ~ ident
def PERCENTAGE = decimalNumber ~ "%"
def NUMBER = decimalNumber | "\\" ~ decimalNumber
def URI = "url(" ~ ((stringLiteral | "[^)]+".r) ^^ (URL(_))) ~ ")"
def hexcolor = "#(?:[0-9A-Fa-f]{3}){1,2}".r
def function = "[a-zA-Z:._0-9-]+\\(".r ~ funcexpr ~ ")"
def unary_operator = "-" | "+"
def term: Parser[Any] = unary_operator | ((PERCENTAGE | LENGTH | EMS | EXS | ANGLE | RESOLUTION |
TIME | FREQ | URI | hexcolor | stringLiteral | NUMBER | ie_expression | function | ident) ^^ (NeedsSpace(_)))
def expr = rep1(term ~ opt(operator))
def ie_expression_no_paren = "[^\\(\\)]+".r
def ie_expression_paren: Parser[Any] = "(" ~ rep(ie_expression_no_paren | ie_expression_paren) ~ ")"
def ie_expression = "expression" ~ ie_expression_paren
// This is an extension of the css spec to allow filter: alpha(opacity=xx) syntax (kwargs).
def funcexpr = rep(opt(ident ~ "=") ~ term ~ opt(operator))
def operator = "/" | ","
def combinator = "+" | ">" | "~"
def prio = IMPORTANT_SYM
def declaration = property ~ ":" ~ expr ~ opt(prio)
def transform_declaration = """(?i)(?:from|to)""".r ~ "{" ~ rep1(declaration ~ rep(";")) ~ "}"
def nth_expr = ("\\d+".r ~ "n" ~ opt(("+" | "-") ~ "\\d+".r)) | (opt("\\d+".r ~ "n") ~ ("+" | "-") ~ "\\d+".r) | "\\d+".r
def pseudo = ":" ~ opt((ident ~ "(" ~ (HASH | class_ | ident | nth_expr | (":" ~ ident)) ~ ")") | ident)
def attrib = "[" ~ ident ~ opt(opt("=" | INCLUDES | DASHMATCH) ~ (ident | stringLiteral)) ~ "]"
def element_name = "*" | ident | "/**/"
def class_ = "." ~ ident
def HASH = "#" ~ ident
def selector_modifier = HASH | class_ | attrib | pseudo
def simple_selector = (element_name ~ rep(selector_modifier)) | (rep1(selector_modifier))
def selector = simple_selector ~ opt(combinator | ",")
def declaration_body = "{" ~ rep(transform_declaration | declaration ~ rep(";")) ~ "}"
def ruleset = rep1(selector ^^ (NeedsSpace(_))) ~ declaration_body
def property = ident
def font_face = FONT_FACE_SYM ~ declaration_body
def moz_document = ("(?i)#-moz-document".r ^^ (NeedsSpace(_))) ~ opt(function) ~ "{" ~ rep(ruleset) ~ "}"
def pseudo_page = ":" ~ ident
def medium = ident
def media_qualifier = "(" ~ ident ~ ":" ~ term ~ ")"
def media_term = (ident | media_qualifier) ~ opt(",")
def page = (PAGE_SYM ^^ (NeedsSpace(_))) ~ opt(ident) ~ opt(pseudo_page) ~ "{" ~ rep1sep(declaration, ",") ~ "}"
def media = (MEDIA_SYM ^^ (NeedsSpace(_))) ~ rep1(media_term) ~ "{" ~ rep(ruleset) ~ "}"
def namespace_prefix = ident
def namespace = (NAMESPACE_SYM ^^ (NeedsSpace(_))) ~ opt(namespace_prefix) ~ opt(stringLiteral | URI) ~ ";"
def import_ = (IMPORT_SYM ^^ (NeedsSpace(_))) ~ (stringLiteral | URI) ~ repsep(medium, ",") ~ ";"
def stylesheet = opt((CHARSET_SYM^^ (NeedsSpace(_))) ~ stringLiteral ~ ";") ~
rep(import_) ~ rep(namespace) ~
rep(media | page | font_face | moz_document | ruleset)
}
case class URL(url: String) {
val AbsolutePattern = """^(?i)(?:/|(?:http|ftp|https|spdy)://).*""".r
val InsideQuote = """^(['\"]?)(.+)\1$""".r
def rewrite(prefix: String): String = url match {
case InsideQuote(quote, content) => {
quote + rewriteInside(content, prefix) + quote
}
case _ => rewriteInside(url, prefix)
}
private def rewriteInside(inside: String, prefix: String): String = inside match {
case AbsolutePattern() => inside
case _ => prefix + inside
}
}
case class NeedsSpace(token: Any)
object Main extends SimpleCSSParser {
var prefix: String = ""
def main(args: Array[String]) {
val noSpace = Set(";", "}", ")", "{", "(", ",", ">", "<", "+")
val targetIndex = args.zipWithIndex filter {case (arg, idx) => arg == "-t"}
val target = if (targetIndex.length > 0)
Some(args(targetIndex(0)._2 + 1))
else
None
val sourceIndex = args.zipWithIndex filter {case (arg, idx) => arg == "-s"}
val sourcePath = if (sourceIndex.length > 0)
Some(args(sourceIndex(0)._2 + 1))
else
None
var originalSourcePath = if (sourcePath.isDefined)
Some(sourcePath.get)
else if (args.length > 0 && args(0) != "-t" && args(0) != "-s")
Some(args(0))
else
None
if (originalSourcePath.isDefined && target.isDefined) {
val parent = new File(originalSourcePath.get).getParent
val sourceList = if (parent == null || parent == "")
Nil
else
parent.split(File.separator).toList
val destParent = new File(target.get).getParent
val destList = if (destParent == null || destParent == "")
Nil
else
destParent.split(File.separator).toList
this.prefix = computePrefix(sourceList, destList).mkString("/")
if (!this.prefix.isEmpty)
this.prefix += "/"
}
val input = if (args.length > 0 && args(0) != "-t" && args(0) != "-s") {
io.Source.fromFile(args(0))
} else {
io.Source.stdin
}
val result = parseAll(stylesheet, input.getLines().mkString("\n"))
try {
val flatResult = flatResultList(result)
// Beautify by removing needless spaces.
flatResult.zipWithIndex foreach { case (value, idx) => if (idx > 0 && noSpace.contains(value) && flatResult(idx - 1) == " ") flatResult(idx - 1) = "" }
print(flatResult.mkString(""))
} catch {
case e: Exception => System.err.println(result)
}
}
#tailrec
def computePrefix(sourceDir: List[String], target: List[String], acc: List[String] = List()):
List[String] = (sourceDir, target) match {
case (shead :: srest, thead :: trest) =>
if (shead == thead)
computePrefix(srest, trest, acc)
else
computePrefix(srest, trest, ".." :: acc ::: List(shead))
case (Nil, thead :: trest) => computePrefix(Nil, trest, ".." :: acc)
case (shead :: srest, Nil) => computePrefix(srest, Nil, acc ::: List(shead))
case (Nil, Nil) => acc
}
def flatResultList(result: Any): ArrayBuffer[String] = result match {
case a: Some[Any] => flatResultList(a.get)
case a: ParseResult[Any] => flatResultList(a.get)
case a: ~[Any, Any] => flatResultList(a._1) ++ flatResultList(a._2)
case a :: rest => flatResultList(a) ++ flatResultList(rest)
case a: String => ArrayBuffer(a)
case None => ArrayBuffer()
case List() => ArrayBuffer()
/* Put any rewrite rule here, and annotate the above tokens with ^^ to do it. */
case url: URL => ArrayBuffer(url.rewrite(this.prefix))
case needsSpace: NeedsSpace => flatResultList(needsSpace.token) ++ ArrayBuffer(" ")
}
}
https://github.com/axiak/scala-css-parser/blob/master/src/testurl.sh
#!/bin/bash
curl "$1" | java -cp /opt/scala/lib/scala-library.jar:. Main
Take a look at Hyperscala (http://www.hyperscala.org). It has an incredibly powerful CSS parser built-in (StyleSheet.parse) that can parse multiple selectors.