I'm playing with AsyncHTTPBuilder (v0.5.1) however, I can't get it working so it's execute requests asynchronously. Please, check the code below. Looks like all requests are done from the same thread:
#Test public void testPoolsizeAndQueueing() {
def http = new AsyncHTTPBuilder( poolSize : 5 ,
uri : 'http://ajax.googleapis.com/ajax/services/search/web' )
def responses = []
/* With one thread in the pool, responses will be sequential but should
* queue up w/o being rejected. */
10.times {
responses << http.get( query : [q:'Groovy', v:'1.0'] ) { return Thread.currentThread().name }
responses << http.get( query : [q:'Ruby', v:'1.0'] ) { return Thread.currentThread().name }
responses << http.get( query : [q:'Scala', v:'1.0'] ) { return Thread.currentThread().name }
}
def timeout = 60000
def time = 0
while ( true ) {
if ( responses.every{ it.done ? it.get() : 0 } ) break
print '.'
Thread.sleep 2000
time += 2000
if ( time > timeout ) assert false
}
responses.each { println it.get() }
http.shutdown()
}
Output:
..pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
pool-3-thread-1
I've fixed the bug and deployed a new 0.5.2-SNAPSHOT. So it will work now with the latest snapshot.
I plan to release 0.5.2 final with this fix and fixes for a number of other bugs soon.
For the record, the named parameter is actually poolSize not poolsize but that's not the cause of the problem. (But this is something I plan to fix in 0.5.2 - http://jira.codehaus.org/browse/GMOD-175 .)
Thanks to Tim Yates for reporting the bug.
I believe this is as expected.
The queries are being run in separate threads asynchronously, but the { return Thread.currentThread().name } closure is getting run when it.get() is called (in the same thread as the main script is running)
Hope this explains it
EDIT
You're right.
Running with
def http = new AsyncHTTPBuilder( poolsize:5,
uri:'http://ajax.googleapis.com/ajax/services/search/web' )
doesn't work as expected...and changing it to:
def http = new AsyncHTTPBuilder( threadPool:java.util.concurrent.Executors.newFixedThreadPool( 5 ),
uri:'http://ajax.googleapis.com/ajax/services/search/web' )
makes it work. Even tried with 0.5.2-SNAPSHOT, but got the same issue
I have reported it as a bug: http://jira.codehaus.org/browse/GMOD-174
Fingers crossed it will get fixed (or we'll find out why we're doing it wrong) by v 0.5.2
Related
I am trying to migrate ClojureScript tests from "Chrome Headless" to jsdom using Karma and shadow-cljs as test runners.
The regular tests which require access to DOM or browser API work fine. But async test where cljs.core.async/go is used doesn't work. Basically, nothing inside go is executed.
Does anyone have some idea what could be wrong? Did I miss some configuration? Is it only jsdom issue or maybe it is cljs.core.async interoperability issue?
I have put a simple test example below
(ns async-tests
(:require [cljs.test :refer [deftest async]]
[cljs.core.async :refer [go <! timeout]]))
(deftest async-go-test
(async done
(.log js/console "Before go is printed")
(go
(.log js/console "After go is never printed")
(<! (timeout 1))
(done))))
The result I get in console is
LOG: 'Testing async-tests'
LOG: 'Before go is printed'
WebKit 537.36 (undefined 0.0.0): Executed 159 of 185 SUCCESS (0 secs / 0.589 secs)
WebKit 537.36 (undefined 0.0.0) ERROR
Disconnected, because no message in 30000 ms.
Versions of libraries which are used:
"devDependencies": {
"jsdom": "^16.4.0",
"karma": "^5.2.3",
"karma-cljs-test": "^0.1.0",
"karma-jsdom-launcher": "^8.0.2",
"shadow-cljs": "2.10.19"
}
Karma configuration:
module.exports = function (config) {
config.set({
browsers: ['jsdom'],
basePath: 'target',
files: ['ci.js'],
frameworks: ['cljs-test'],
colors: true,
logLevel: config.LOG_INFO,
client: {
args: ["shadow.test.karma.init"]
},
jsdomLauncher: {
jsdom: {
resources: "usable",
runScripts: "dangerously",
pretendToBeVisual: true
}
}
})
};
I would like to say "Thank you" to the Clojure community which helped me find a workaround for this issue and especially #thheller.
The root cause is not found yet but probably that it is the result of using the
range of different libraries.
Workaround
You have to override goog.async.nextTick method for your tests with
js/setTimeout
Example
Create jsdom-setup namespace
(ns jsdom-setup)
(set! (.. js/window -goog -async -nextTick) js/setTimeout)
and add it to the tests JS output inside 'shadow-cljs.edn'
:builds {:karma {:target :karma
:output-to "output/tests-bundle.js"
:ns-regexp "(setup-jsdom|[.-]tests$)"}}
Links
Clojure Slack discussion
I cannot for the life of me get the LocalAttestation sample to run correctly on a fresh Linux install, following the instructions successfully. Given this is being built in simulation mode, I would have thought there were no additional dependencies?
I modified the demo to provide extra output, and this line returns 3002 SGX_ERROR_INVALID_ATTRIBUTE:
printf("creating enclave 1\n");
ret = sgx_create_enclave(ENCLAVE1_PATH, SGX_DEBUG_FLAG, &launch_token, &launch_token_updated, &e1_enclave_id, NULL);
if (ret != SGX_SUCCESS) {
printf("Failed. Return value is: %X\n", ret);
return ret;
}
This is the sample, right out of the linux SDK: https://github.com/intel/linux-sgx and this error isn't even mentioned as a possibility in Intel's documentation on the function: https://software.intel.com/en-us/sgx-sdk-dev-reference-sgx-create-enclave
Any help would be greatly appreciated!
-- Henry
Turns out this was an issue with the sample code. By initializing the launch_token zeroed out, everything works as expected:
sgx_launch_token_t launch_token = {0};
ret =sgx_create_enclave(_T(ENCLAVE_PATH),1,&launch_token,&launch_token_update,&enclave_id, NULL);
if(ret !=SGX_SUCCESS)
{
printf("Failed to create enclave");
}
printf("Successfully create enclave.");
printf("\nEnclaveID %llx", enclave_id);
I'm trying to understand how to properly setup JavaFX to work with a Clojure project. By reading various sources this is what I've come up with:
This is project.clj:
(defproject cljfx "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.8.0"]]
:resource-paths ["lib/jfxrt.jar"]
:main ^:skip-aot cljfx.core
:target-path "target/%s"
:profiles {:uberjar {:aot :all}})
I don't know if I should use :resource-paths or add JavaFX to the classpath via the :dependencies vector...
This is core.clj:
I've basically translated to Clojure an example from this tutorial:
http://docs.oracle.com/javafx/2/get_started/hello_world.htm
(ns cljfx.core
(:gen-class
:extends javafx.application.Application)
(:import
[javafx.application Application]
[javafx.stage Stage]
[javafx.scene Scene]
[javafx.scene.control Button]
[javafx.scene.layout StackPane]
[javafx.event ActionEvent EventHandler]))
(defn -main [& args]
(Application/launch cljfx.core args))
(defn button [text]
(doto (Button.)
(.setText (str "Say " text))
(.setOnAction (proxy [EventHandler] []
(handle [event]
(println text))))))
(defn -start [primaryStage]
(let [root (doto (StackPane.)
(-> (.getChildren)
(.add (button "Hello World!"))))]
(doto primaryStage
(.setScene (Scene. root 300 250))
(.show))))
This doesn't compile, and I don't know what I'm doing wrong... can you help me?
Here is the error:
http://pastebin.com/sYeK7MJd
There may be other problems, but the root problem in the pastebin log is:
Caused by: clojure.lang.ArityException: Wrong number of args (2) passed to: core/-start
When using gen-class and providing method implementations, every method needs to take the instance itself as the first parameter. The convention is to use "this":
(defn -start [this primaryStage]
Try that, and ensure that local instance method calls are applied to "this".
I have a piece of code as show belowe, wich run well with C++Builder-6.
Now I have moved tha program to C++Builder-XE and the call to "RiconfiguraNodo << nomeNodo ...." give me the ambguity error report belowe.
I tried several way to rewrite the call to the ole proceudre "RiconfiguraNodo", but I didn't find a working solution.
How can I rewrite this snippet of code in way suitable for C++BuilderXE
Error reported:
[BCC32 Error] UnitMain.cpp(262): E2015 Ambiguity between 'operator
System::AutoCmd::<<(const System::Currency) at c:\program files
(x86)\embarcadero\rad studio\8.0\include\windows\rtl\sysvari.h:3561'
and 'operator System::AutoCmd::<<(const System::TDateTime)
at c:\program files (x86)\embarcadero\rad
studio\8.0\include\windows\rtl\sysvari.h:3562'
Full parser context
UnitMain.cpp(245): parsing: void _fastcall TFormMain::RiconfiguraNodo(System::UnicodeString,System::UnicodeString,System::UnicodeString,System::UnicodeString)
Sample code:
Procedure RiconfiguraNodo( L"RiconfiguraNodo" );
if (VarServerPmvManager.IsEmpty() || VarServerPmvManager.IsNull())
{
VarServerPmvManager = VarServerPmvManager.CreateObject(ProgId_ServerPmvmanager);
}
try
{
VarServerPmvManager.Exec( RiconfiguraNodo << nomeNodo << ipAddress << tipoPmv << cmdType );
}
catch (Exception & ex)
{
Mylog(Fun + Sysutils::Format("ERROR=[%s] ", ARRAYOFCONST((ex.Message))));
}
I found the solution.
The procedure exec simply require Variant instead of plain string
Variant vNomeNodo, vIpAddress, vTipoPmv, vCmdType;
vNomeNodo = nomeNodo;
vIpAddress = ipAddress;
vTipoPmv = tipoPmv;
vCmdType = cmdType;
VarServerPmvManager.Exec( RiconfiguraNodo << vNomeNodo << vIpAddress << vTipoPmv << vCmdType );
I try to switch an existing crawler from EventMachine to Celluloid. To get in touch with Celluloid I've generated a bunch of static files with 150 kB per file on a linux box which are served via Nginx.
The code at the bottom should do its work, but there is a issues with the code which I don't understand: the code should spawn maximum 50 threads because of the thread pool size of 50 but it spawns 180 of them. If I increase the pool size to 100, 330 threads are spawned. What's going wrong there?
A simple copy & paste of this code should work on every box, so any hints are welcome :)
#!/usr/bin/env jruby
require 'celluloid'
require 'open-uri'
URLS = *(1..1000)
##requests = 0
##responses = 0
##total_size = 0
class Crawler
include Celluloid
def fetch(id)
uri = URI("http://data.asconix.com/#{id}")
puts "Request ##{##requests += 1} -> #{uri}"
begin
req = open(uri).read
rescue Exception => e
puts e
end
end
end
URLS.each_slice(50).map do |idset|
pool = Crawler.pool(size: 50)
crawlers = idset.to_a.map do |id|
begin
pool.future(:fetch, id)
rescue Celluloid::DeadActorError, Celluloid::MailboxError
end
end
crawlers.compact.each do |resp|
$stdout.print "Response ##{##responses += 1} -> "
if resp.value.size == 150000
$stdout.print "OK\n"
##total_size += resp.value.size
else
$stdout.print "ERROR\n"
end
end
pool.terminate
puts "Actors left: #{Celluloid::Actor.all.to_set.length} -- Alive: #{Celluloid::Actor.all.to_set.select(&:alive?).length}"
end
$stdout.print "Requests total: #{##requests}\n"
$stdout.print "Responses total: #{##responses}\n"
$stdout.print "Size total: #{##total_size} bytes\n"
By the way, the same issue occurs when I define the pool outside the each_slice loop:
....
#pool = Crawler.pool(size: 50)
URLS.each_slice(50).map do |idset|
crawlers = idset.to_a.map do |id|
begin
#pool.future(:fetch, id)
rescue Celluloid::DeadActorError, Celluloid::MailboxError
end
end
crawlers.compact.each do |resp|
$stdout.print "Response ##{##responses += 1} -> "
if resp.value.size == 150000
$stdout.print "OK\n"
##total_size += resp.value.size
else
$stdout.print "ERROR\n"
end
end
puts "Actors left: #{Celluloid::Actor.all.to_set.length} -- Alive: #{Celluloid::Actor.all.to_set.select(&:alive?).length}"
end
What ruby are you using? jRuby, Rubinius, etc? And what version of those?
Reason I ask is, threads are handled differently for each ruby. What you seem to be describing is added threads that come in for supervisors, and for tasks. Looking at the date of your post, it is likely that fibers are actually becoming native threads, which would make it seem like using jRuby perhaps. Also, using Futures often invokes the internal thread pool, which has nothing to do with your pool.
With both those reasons and others like them you could look for, that makes sense why you'd have a higher thread count than your pool calls for. This is a bit old though, so maybe you could follow up as to whether you still have this problem, and post output.