Downloading artifact: CompilerSupportLibraries
curl: (52) Empty reply from server
ERROR: LoadError: Unable to automatically install 'CompilerSupportLibraries' from '/home/zyh/.julia/packages/CompilerSupportLibraries_jll/790hI/Artifacts.toml'
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] ensure_artifact_installed(::String, ::Dict{String,Any}, ::String; platform::Pkg.BinaryPlatforms.Platform, verbose::Bool, quiet_download::Bool) at /usr/local/julia/julia-1.4.2/usr/share/julia/stdlib/v1.4/Pkg/src/Artifacts.jl:894
[3] ensure_all_artifacts_installed(::String; platform::Pkg.BinaryPlatforms.Platform, pkg_uuid::Nothing, include_lazy::Bool, verbose::Bool, quiet_download::Bool) at /usr/local/julia/julia-1.4.2/usr/share/julia/stdlib/v1.4/Pkg/src/Artifacts.jl:958
[4] download_artifacts(::Pkg.Types.Context, ::Array{String,1}; platform::Pkg.BinaryPlatforms.Linux, verbose::Bool) at /usr/local/julia/julia-1.4.2/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:616
[5] download_artifacts(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; platform::Pkg.BinaryPlatforms.Linux, verbose::Bool) at /usr/local/julia/julia-1.4.2/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:595
[6] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Array{Base.UUID,1}; preserve::Pkg.Types.PreserveLevel, platform::Pkg.BinaryPlatforms.Linux) at /usr/local/julia/julia-1.4.2/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:1091
[7] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; preserve::Pkg.Types.PreserveLevel, platform::Pkg.BinaryPlatforms.Linux, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /usr/local/julia/julia-1.4.2/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:159
[8] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /usr/local/julia/julia-1.4.2/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:112
[9] #add#27 at /usr/local/julia/julia-1.4.2/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:109 [inlined]
[10] add at /usr/local/julia/julia-1.4.2/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:109 [inlined]
[11] #add#23 at /usr/local/julia/julia-1.4.2/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:106 [inlined]
[12] add(::Pkg.Types.PackageSpec) at /usr/local/julia/julia-1.4.2/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:106
[13] top-level scope at /home/zyh/DOLPHYN_old/julenv.jl:20
[14] include(::String) at ./client.jl:439
[15] top-level scope at REPL[1]:1
I'm trying to construct a specific environment using JuMP 0.18.6 in Julia 1.4.2. But when downloading JuMP, it seems that this problem got blocking my may. Had anybody ever encountered this before and have a solution?
it's likely due to firewall of some sort, if you're from China, try the mirror source:
https://mirrors.tuna.tsinghua.edu.cn/help/julia/
I work mostly on offline machines and really want to begin to migrate from Python to Julia. Currently the biggest problem I face is how can I setup a package server on my own network that does not have access to the internet. I can copy files to the offline computer/network and want to be able to just cache a good percentage of the Julia Package Ecosystem and copy it to my network, so that I and others can install packages as needed.
I have experimented with PkgSever.jl by using the deployment docker-compose script they have, then just installing a long list of packages so that the PkgServer instance would cache everything. Next took the PkgServer machine offline and attempted to install packages from it. This worked well, however when I restarted the docker container the server was running in, everything fell apart quickly.
It seems that maybe the PkgServer needs to be able to talk to the Storage Server at least once before being able to serve packages. I tried setting:
JULIA_PKG_SERVER_STORAGE_SERVERS from: "https://us-east.storage.juliahub.com,https://kr.storage.juliahub.com" to: "" but that failed miserably.
Can someone please point me in the right direction.
TIA
It looks like the PkgServer is actually trying to contact the Registry before it starts. I don't know enough about the registry stuff enough to know if there is a way to hack this to look locally or just ignore this..
pkgserver_1 | ERROR: LoadError: DNSError: kr.storage.juliahub.com, temporary failure (EAI_AGAIN)
pkgserver_1 | Stacktrace:
pkgserver_1 | [1] getalladdrinfo(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Sockets/src/addrinfo.jl:112
pkgserver_1 | [2] getalladdrinfo at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Sockets/src/addrinfo.jl:121 [inlined]
pkgserver_1 | [3] getconnection(::Type{TCPSocket}, ::SubString{String}, ::String; keepalive::Bool, connect_timeout::Int64, kw::Base.Iterators.Pairs{Symbol,Union{Nothing, Bool},NTuple{4,Symbol},NamedTuple{(:require_ssl_verification, :iofunction, :reached_redirect_limit, :status_exception),Tuple{Bool,Nothing,Bool,Bool}}}) at /depot/packages/HTTP/IAI92/src/ConnectionPool.jl:630
pkgserver_1 | [4] #getconnection#29 at /depot/packages/HTTP/IAI92/src/ConnectionPool.jl:682 [inlined]
pkgserver_1 | [5] newconnection(::HTTP.ConnectionPool.Pod, ::Type{T} where T, ::SubString{String}, ::SubString{String}, ::Int64, ::Bool, ::Int64; kw::Base.Iterators.Pairs{Symbol,Union{Nothing, Bool},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:iofunction, :reached_redirect_limit, :status_exception),Tuple{Nothing,Bool,Bool}}}) at /depot/packages/HTTP/IAI92/src/ConnectionPool.jl:597
pkgserver_1 | [6] getconnection(::Type{HTTP.ConnectionPool.Transaction{MbedTLS.SSLContext}}, ::SubString{String}, ::SubString{String}; connection_limit::Int64, pipeline_limit::Int64, idle_timeout::Int64, reuse_limit::Int64, require_ssl_verification::Bool, kw::Base.Iterators.Pairs{Symbol,Union{Nothing, Bool},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:iofunction, :reached_redirect_limit, :status_exception),Tuple{Nothing,Bool,Bool}}}) at /depot/packages/HTTP/IAI92/src/ConnectionPool.jl:541
pkgserver_1 | [7] request(::Type{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer{Union{}}}}, ::HTTP.URIs.URI, ::HTTP.Messages.Request, ::Array{UInt8,1}; proxy::Nothing, socket_type::Type{T} where T, reuse_limit::Int64, kw::Base.Iterators.Pairs{Symbol,Union{Nothing, Bool},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:iofunction, :reached_redirect_limit, :status_exception),Tuple{Nothing,Bool,Bool}}}) at /depot/packages/HTTP/IAI92/src/ConnectionRequest.jl:73
pkgserver_1 | [8] (::Base.var"#56#58"{Base.var"#56#57#59"{ExponentialBackOff,HTTP.RetryRequest.var"#2#3"{Bool,HTTP.Messages.Request},typeof(HTTP.request)}})(::Type{T} where T, ::Vararg{Any,N} where N; kwargs::Base.Iterators.Pairs{Symbol,Union{Nothing, Bool},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:iofunction, :reached_redirect_limit, :status_exception),Tuple{Nothing,Bool,Bool}}}) at ./error.jl:301
pkgserver_1 | [9] #request#1 at /depot/packages/HTTP/IAI92/src/RetryRequest.jl:44 [inlined]
pkgserver_1 | [10] request(::Type{HTTP.MessageRequest.MessageLayer{HTTP.RetryRequest.RetryLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer{Union{}}}}}}, ::String, ::HTTP.URIs.URI, ::Array{Pair{SubString{String},SubString{String}},1}, ::Array{UInt8,1}; http_version::VersionNumber, target::String, parent::Nothing, iofunction::Nothing, kw::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol,Symbol},NamedTuple{(:reached_redirect_limit, :status_exception),Tuple{Bool,Bool}}}) at /depot/packages/HTTP/IAI92/src/MessageRequest.jl:51
pkgserver_1 | [11] request(::Type{HTTP.BasicAuthRequest.BasicAuthLayer{HTTP.MessageRequest.MessageLayer{HTTP.RetryRequest.RetryLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer{Union{}}}}}}}, ::String, ::HTTP.URIs.URI, ::Array{Pair{SubString{String},SubString{String}},1}, ::Array{UInt8,1}; kw::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol,Symbol},NamedTuple{(:reached_redirect_limit, :status_exception),Tuple{Bool,Bool}}}) at /depot/packages/HTTP/IAI92/src/BasicAuthRequest.jl:28
pkgserver_1 | [12] request(::Type{HTTP.RedirectRequest.RedirectLayer{HTTP.BasicAuthRequest.BasicAuthLayer{HTTP.MessageRequest.MessageLayer{HTTP.RetryRequest.RetryLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer{Union{}}}}}}}}, ::String, ::HTTP.URIs.URI, ::Array{Pair{SubString{String},SubString{String}},1}, ::Array{UInt8,1}; redirect_limit::Int64, forwardheaders::Bool, kw::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol},NamedTuple{(:status_exception,),Tuple{Bool}}}) at /depot/packages/HTTP/IAI92/src/RedirectRequest.jl:24
pkgserver_1 | [13] request(::String, ::String, ::Array{Pair{SubString{String},SubString{String}},1}, ::Array{UInt8,1}; headers::Array{Pair{SubString{String},SubString{String}},1}, body::Array{UInt8,1}, query::Nothing, kw::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol},NamedTuple{(:status_exception,),Tuple{Bool}}}) at /depot/packages/HTTP/IAI92/src/HTTP.jl:314
pkgserver_1 | [14] #get#12 at /depot/packages/HTTP/IAI92/src/HTTP.jl:391 [inlined]
pkgserver_1 | [15] get_registries(::String) at /app/src/resource.jl:21
pkgserver_1 | [16] update_registries() at /app/src/resource.jl:130
pkgserver_1 | [17] start(; kwargs::Base.Iterators.Pairs{Symbol,Any,Tuple{Symbol,Symbol,Symbol},NamedTuple{(:listen_addr, :storage_root, :storage_servers),Tuple{Sockets.InetAddr{IPv4},String,Array{SubString{String},1}}}}) at /app/src/PkgServer.jl:88
pkgserver_1 | [18] top-level scope at /app/bin/run_server.jl:43
pkgserver_1 | [19] include(::Function, ::Module, ::String) at ./Base.jl:380
pkgserver_1 | [20] include(::Module, ::String) at ./Base.jl:368
pkgserver_1 | [21] exec_options(::Base.JLOptions) at ./client.jl:296
pkgserver_1 | [22] _start() at ./client.jl:506
pkgserver_1 | in expression starting at /app/bin/run_server.jl:43
This might be helpful but I'm not sure, yet, how to get it started
LocalRegistry.jl
Here is a solution that seems to work, based on LocalPackageServer.
Preliminary steps
Install all required packages. You can either put them in your default environment (e.g. #v1.5) or in a dedicated project.
LocalRegistry
LocalPackageServer
In order to use LocalPackageServer, we'll need to set up a local registry, even though we won't really use it (but it can still be handy if you also have to serve local packages).
Something like this should create an empty local registry as local-registry.gitt in the current folder:
# Create an empty (bare) git repository to host the registry
run(`git init --bare local-registry.git`)
# Populate the repository with a new, empty local registry
using LocalRegistry
Base.Filesystem.mktempdir() do dir
create_registry(joinpath(dir, "local-registry"),
abspath("local-registry.git"),
description = "(unused) local registry",
push=true)
end
Step 1a - run the local package server (online)
A script like the following should run a local package server listening on http://localhost:8000.
#################
# run_server.jl #
#################
using LocalPackageServer
config = LocalPackageServer.Config(Dict(
# Server parameters
"host" => "127.0.0.1",
"port" => 8000,
"pkg_server" => "https://pkg.julialang.org",
# This is where persistent data will be stored
# (I use the current directory here; adjust to your constraints)
"local_registry" => abspath("local-registry.git"), # In accordance with the preliminary step above
"cache_dir" => abspath("cache"),
"git_clones_dir" => abspath("data"),
))
# The tricky part: arrange for the server to never update its registries
# when it is offline
if get(ENV, "LOCAL_PKG_SERVER_OFFLINE", "0") == "1"
#info "Running offline => no registry updates"
config.min_time_between_registry_updates = typemax(Int)
end
# Start the server
LocalPackageServer.start(config)
Use this script to run the server online first:
shell$ julia --project run_server.jl
Step 1b - cache some packages (online)
Configure a Julia process to use your local server, and install the packages you want to cache:
# Take care to specify the "http://" protocol
# (otherwise https might be used by the client, and the server won't answer)
shell$ JULIA_PKG_SERVER=http://localhost:8000 julia
julia> using Pkg
julia> pkg"add Example"
[...]
At this point, the server should be caching things.
Step 2 - run the package server (offline)
When you're offline, simply restart the server, ensuring it won't try to update the registries:
shell$ LOCAL_PKG_SERVER_OFFLINE=1 julia --project run_server.jl
As before, set JULIA_PKG_SERVER as needed for Julia clients to use the local server; they should now be able to install packages, provided that the project environments resolve to the exact same dependency versions that were cached.
(You might want to resolve and instantiate your project environments online, and then transfer the relevant manifests to offline systems: this might help guarantee the consistency between the package server cache and what clients ask for.)
Trying to understand following error in a pmap execution of some embarrassingly parallel tasks. Running on Linux server. might be occurring while writing to HDF (part of parallel call), but I don't think so given stacktrace doesn't point to line in the user-function being executed, and reference to TCP suggests it's part of parallel calls. Has happened in several sequential runs, so not one-time fluke.
Worker 139 terminated.
ERROR (unhandled task failure): EOFError: read end of file
Stacktrace:
[1] unsafe_read(::Base.AbstractIOBuffer{Array{UInt8,1}}, ::Ptr{UInt8}, ::UInt64) at ./iobuffer.jl:105
[2] unsafe_read(::TCPSocket, ::Ptr{UInt8}, ::UInt64) at ./stream.jl:752
[3] unsafe_read(::TCPSocket, ::Base.RefValue{NTuple{4,Int64}}, ::Int64) at ./io.jl:361
[4] read at ./io.jl:363 [inlined]
[5] deserialize_hdr_raw at ./distributed/messages.jl:170 [inlined]
[6] message_handler_loop(::TCPSocket, ::TCPSocket, ::Bool) at ./distributed/process_messages.jl:157
[7] process_tcp_streams(::TCPSocket, ::TCPSocket, ::Bool) at ./distributed/process_messages.jl:118
[8] (::Base.Distributed.##99#100{TCPSocket,TCPSocket,Bool})() at ./event.jl:73
Julia info:
julia> versioninfo()
Julia Version 0.6.0
Commit 9036443 (2017-06-19 13:05 UTC)
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
CPU: Intel(R) Xeon(R) CPU E5620 # 2.40GHz
WORD_SIZE: 64
BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Nehalem)
LAPACK: libopenblas64_
LIBM: libopenlibm
LLVM: libLLVM-3.9.1 (ORCJIT, westmere)
[EDIT: more info] Also, if it's helpful, this appears to be happening well into the run -- output from the first set of parallel runs looks like it's being saved to disk, so this is not an immediate crash, but something that's happening at the end of a run or start of the second set of executions.
OK, so I did finally figure out at a high level what this means:
This is the error you get when one of the parallelized workers hits an error. The specific error language (EOFError: read end of file) doesn't really mean anything. And the references to read and io in the stackoverflow just relate to messaging between the overview task and workers.
In my case, the error was a memory overflow leading to the task manager terminating the worker.
I have implemented the IT Hit WebDAV server on our company website. I've been looking at the logs and I can see it unlocks PowerPoint presentation twice. Once just before it open and then when you close MS PowerPoint.
Can I stop this from happening so when you open a PowerPoint document it unlock only when the document closes like MS Word.
When I open a Word document it only unlocks it once when you close the MS Word.
As you can see from the logs below there are less request in MS Word than in MS PowerPoint. Both documents I followed the same process:
Open
Edit
Save
Close
The reason why I would like it to unlock only once is so I can implement some custom code for security which should only run when the user has finished using the document.
Microsoft Word
[29] [OPTIONS] /DAV/
[45] [HEAD] /DAV/437f144e-c42a-4e8d-97b2-45fa3d1f0a71/Document.docx
[99] [OPTIONS] /DAV/
[79] [LOCK] /DAV/437f144e-c42a-4e8d-97b2-45fa3d1f0a71/Document.docx
[99] [GET] /DAV/437f144e-c42a-4e8d-97b2-45fa3d1f0a71/Document.docx
[54] [PROPFIND] /DAV/437f144e-c42a-4e8d-97b2-45fa3d1f0a71/Document.docx
[74] [LOCK] /DAV/437f144e-c42a-4e8d-97b2-45fa3d1f0a71/Document.docx
[94] [PUT] /DAV/437f144e-c42a-4e8d-97b2-45fa3d1f0a71/Document.docx
[94] [UNLOCK] /DAV/437f144e-c42a-4e8d-97b2-45fa3d1f0a71/Document.docx
PowerPoint
[89] [OPTIONS] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/
[86] [HEAD] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[89] [OPTIONS] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/
[86] [LOCK] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[89] [GET] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[97] [PROPFIND] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[65] [HEAD] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[68] [UNLOCK] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[97] [OPTIONS] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/
[86] [HEAD] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[97] [GET] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[100] [PROPFIND] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[68] [HEAD] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[86] [LOCK] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[89] [GET] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[68] [PROPFIND] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[97] [HEAD] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[59] [LOCK] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[59] [PUT] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
[86] [UNLOCK] /DAV/c763764d-3ba2-46f1-abee-07fa33241309/PowerPoint.pptx
I guess PowerPoint displayed the "Protected View" yellow ribbon on top with "Enable Editing" button, while Word - did not.
This is probably because your Word document was empty (0 bytes), while the PowerPoint - not.
Try the following and you will get 2 locks with Word file:
Create the word file in your local file system, edit it and save, so
it is not 0-bytes.
Upload it to your WebDAV server.
Open it for editting.
Another possible reason - you already opened Word document on this computer in the past (so the "Protected View" did not activate), while the PowerPoint document was opened for the firs time.
MS Office 2013 locks the document when it is being opened. In case it activates "Protected View" the document is unlocked immediately after opening. If you click "Enable Editing" the document is locked again. It is unlocked when the user closes document or when the lock token expires.
In general there should be no problem in multiple locking and unlocking the document, they always come in pairs, as in your log.
Please also note that locking is requested for limited period of time. In case the MS Office needs a longer lock it will prolong the lock, the server will call ILock.RefreshLock in this case.