I want to help Julia find my .jl file by modifying the LOAD_PATH variable:
julia> readdir()
1-element Array{String,1}:
"test.jl"
shell> cmd /c type test.jl
# Test.jl
module Test
export f
f() = println("hi")
end
julia> push!(LOAD_PATH,pwd());
julia> import Test
ERROR: ArgumentError: Module Test not found in current path.
Run `Pkg.add("Test")` to install the Test package.
in require(::Symbol) at .\loading.jl:365
The first call to readdir() proves that I have a file called test.jl in my current directory. The following shell call shows that this file contains a module called Test. The next call to push!(LOAD_PATH,pwd()); puts the current directory in LOAD_PATH. But even with the current directory in LOAD_PATH, Julia still can't find the Test module in test.jl.
What's wrong?
The error was talking about something concerning require. As the doc says:
Given the statement using Foo, the system looks for Foo within Main. If the module does not exist, the system attempts to require("Foo"), which typically results in loading code from an installed package. ... require is case-sensitive on all platforms, including those with case-insensitive filesystems like macOS and Windows.
and the reason is clear: require couldn't find a file named Test in LOAD_PATH. So we need to make the file name matching the module name, but this is just a convention, not a mandatory rule. What will happen if someone mistakenly runs using test?
julia> push!(LOAD_PATH,pwd())
julia> using test
WARNING: requiring "test" in module "Main" did not define a corresponding module.
julia> whos()
Base 34427 KB Module
Core 12386 KB Module
Main 41296 KB Module
Test 1837 bytes Module
The result shows that we've loaded the file test.jl and the module(Test) in it, but not actually using/import the module. This is a respected behavior since we used a wrong module name, which is also the reason why julia complained in the warning. In this case, using test is equivalent to include("test.jl"), but I highly recommend you to follow the convention and do not use this behavior.
BTW, require became generally case-sensitive after this PR. A side-effect is your LOAD_PATH should also be case-sensitive, this will be fixed by this PR in julia-0.6.
Related
I am trying to run Julia unit tests from the command line but the unit tests fail to run because they cannot find a dependency that I am using in my main project. How can I make this work? The actual command that I try to execute is julia test/test_blueprint.jl from the project root. Here follows more details.
Details about the setup
My project is located at the path /home/jonas/prog/julia/blueprint. In that directory, I have a Project.toml file containing these lines:
name = "blueprint"
uuid = "c1615a0c-c255-402d-ae34-0b88819b43c6"
authors = [""]
version = "0.1.0"
[deps]
FunctionalCollections = "de31a74c-ac4f-5751-b3fd-e18cd04993ca"
Setfield = "efcf1570-3423-57d1-acb7-fd33fddbac46"
along with the Manifest.toml file.
I have a subdirectory at test/ with unit tests that I created following this guide and that directory contains another Project.toml file containing
[deps]
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
There is a file test/test_blueprint.jl with unit tests and that file starts with
using Test
include("../src/blueprint.jl") # Alternative 1
#using blueprint # Alternative 2
using FunctionalCollections
using LinearAlgebra
...
The actual code being tested is in the file src/blueprint.jl.
Details about the problem
From the project root, I attempt to run the unit tests using the command julia test/test_blueprint.jl. When I run that command it produces the following output:
ERROR: LoadError: ArgumentError: Package Setfield not found in current path:
- Run `import Pkg; Pkg.add("Setfield")` to install the Setfield package.
Stacktrace:
[1] require(into::Module, mod::Symbol)
# Base ./loading.jl:967
[2] include(fname::String)
# Base.MainInclude ./client.jl:451
[3] top-level scope
# ~/prog/julia/blueprint/test/test_blueprint.jl:8
in expression starting at /home/jonas/prog/julia/blueprint/src/blueprint.jl:1
in expression starting at /home/jonas/prog/julia/blueprint/test/test_blueprint.jl:8
suggesting that it cannot find the dependency Setfield. If I edit the top of the file test/test_blueprint.jl slightly from
include("../src/blueprint.jl") # Alternative 1
#using blueprint # Alternative 2
to
#include("../src/blueprint.jl") # Alternative 1
using blueprint # Alternative 2
it still fails, but with a different error:
ERROR: LoadError: ArgumentError: Package blueprint not found in current path:
- Run `import Pkg; Pkg.add("blueprint")` to install the blueprint package.
Stacktrace:
[1] require(into::Module, mod::Symbol)
# Base ./loading.jl:967
in expression starting at /home/jonas/prog/julia/blueprint/test/test_blueprint.jl:9
Question: How can I make the unit tests run from the command line?
Note that I can run the unit tests from within the Julia REPL in Emacs by activating the project using C-c C-a at the src/blueprint.jl file and calling C-c C-b at the unit test file test/test_blueprint.jl. My Julia version is 1.7.0 (2021-11-30). Don't hesitate to ask for more clarifications.
First, a few naming conventions that are probably not (but may be) contributing to the issues here:
By convention, package names begin with a single capital, so I would recommend changing the name to Blueprint everywhere
By default, ] test runs tests found in the test/runtests.jl, so I would recommend naming your top-level testing script runtests.jl to avoid confusion, even though it does seem from the errors here that test is finding your test_blueprint.jl file one way or another.
Now, while I can't test this without the full code of your package, what I suspect is happening here is the following:
Normally, dependencies of the package you are testing (let's say MyPackage) are not required in test/Project.toml because they are implicit in MyPackage. So after a successful using MyPackage, while they will still not be available to any functions written in your test scripts (test/runtests.jl), will be available to the functions written in MyPackage -- just as if you had typed ]using MyPackage at the REPL and then run your test code there. This is the only reason you don't normally need to duplicate all the deps from the main Project.toml in test/Project.toml.
Since the using Blueprint approach is failing here for other reasons, when you simply include the code from src/blueprint.jl, the usings within that file will in turn fail because those packages are not present in the active environment at test/Project.toml (even if they are present on your system elsewhere).
Consequently, one quick fix to your problem with the current include("../src/blueprint.jl") approach would be to simply add those dependencies to your test/Project.toml
However, it would be more satisfying to fix the problem you are having with using Blueprint. I don't have enough information to debug this without seeing the full structure of your packages, but I would suggest as a start
making sure that your code is properly structured as a package
testing that, even if unregistered, you can ] add your package from the REPL by git repo URL (i.e. ] add https://some_website.com/you/Blueprint.jl)
EDIT:
Upon inspection of the code linked in the comments (https://github.com/jonasseglare/Blueprint), a few other issues:
Although they are already installed by default, standard libraries these days do need to be included in [deps]. In this case, that means the LinearAlgebra stdlib
Any packages you are explicitly using in your test scripts, other than your package itself, do need to be added to test/Project.toml. I.e., any packages that you are directly using functions from in your test scripts (rather than just indirectly using via the exported functions of your package) do need to be included in test/Project.toml.
In your case, the latter would appear to mean LinearAlgebra and FunctionalCollections, but not Setfield (that one only needs to be included in the regular Project.toml, since it's not being directly used in runtests.jl).
Consequently, with a few minor changes to your repo we are able to simply
] add https://github.com/brenhinkeller/Blueprint
] test Blueprint
or, since you preferred at the command line
user$ julia -e "using Pkg; Pkg.add(url=\"https://github.com/brenhinkeller/Blueprint\")
user$ julia -e "using Pkg; Pkg.test(\"Blueprint\")"
Testing Blueprint
Status `/private/var/folders/qk/2qyrdb854mvd2tn4crc802lw0000gn/T/jl_fSypP7/Project.toml`
[c1615a0c] Blueprint v0.1.0 `https://github.com/brenhinkeller/Blueprint#master`
[de31a74c] FunctionalCollections v0.5.0
[37e2e46d] LinearAlgebra `#stdlib/LinearAlgebra`
[8dfed614] Test `#stdlib/Test`
Status `/private/var/folders/qk/2qyrdb854mvd2tn4crc802lw0000gn/T/jl_fSypP7/Manifest.toml`
[c1615a0c] Blueprint v0.1.0 `https://github.com/brenhinkeller/Blueprint#master`
[187b0558] ConstructionBase v1.3.0
[de31a74c] FunctionalCollections v0.5.0
[1914dd2f] MacroTools v0.5.9
[ae029012] Requires v1.3.0
[efcf1570] Setfield v0.8.1
[56f22d72] Artifacts `#stdlib/Artifacts`
[2a0f44e3] Base64 `#stdlib/Base64`
[9fa8497b] Future `#stdlib/Future`
[b77e0a4c] InteractiveUtils `#stdlib/InteractiveUtils`
[8f399da3] Libdl `#stdlib/Libdl`
[37e2e46d] LinearAlgebra `#stdlib/LinearAlgebra`
[56ddb016] Logging `#stdlib/Logging`
[d6f4376e] Markdown `#stdlib/Markdown`
[9a3f8284] Random `#stdlib/Random`
[ea8e919c] SHA `#stdlib/SHA`
[9e88b42a] Serialization `#stdlib/Serialization`
[8dfed614] Test `#stdlib/Test`
[cf7118a7] UUIDs `#stdlib/UUIDs`
[e66e0078] CompilerSupportLibraries_jll `#stdlib/CompilerSupportLibraries_jll`
[4536629a] OpenBLAS_jll `#stdlib/OpenBLAS_jll`
[8e850b90] libblastrampoline_jll `#stdlib/libblastrampoline_jll`
Testing Running tests...
Test Summary: | Pass Total
Plane tests | 7 7
Test Summary: | Pass Total
Plane intersection | 2 2
Test Summary: | Pass Total
Plane intersection 2 | 4 4
Test Summary: | Pass Total
Plane shadowing | 3 3
Test Summary: | Pass Total
Polyhedron tests | 3 3
Test Summary: | Pass Total
Polyhedron tests 2 | 5 5
Test Summary: | Pass Total
Beam tests | 2 2
Test Summary: | Pass Total
Half-space test | 2 2
Test Summary: | Pass Total
Ordered pair test | 2 2
Test Summary: | Pass Total
Test plane/line intersection | 2 2
Test Summary: | Pass Total
Update line bounds test | 21 21
Testing Blueprint tests passed
FWIW, you should also be able to mix and match those command-line and REPL approaches (i.e., install in repl, test via command line or vice versa).
While I had not originally considered this case, one additional possibility discussed in the comments is where one wishes to test the local state of a package without, or without relying upon, a git remote; in this case #Rulle reports that activating the package directory, i.e,
julia -e "using Pkg; Pkg.activate(\".\"); Pkg.test(\"Blueprint\")"
or
julia --project=. -e "using Pkg; Pkg.test(\"Blueprint\")"
or equivalently in the REPL
] activate .
] test Blueprint
will work assuming the package directory is currently the local directory .
Possible answer to my own question:
To make it work, specify the main project root directory on the command line when calling the script using --project. In this case, we would call
julia --project=/home/jonas/prog/julia/blueprint test/test_blueprint.jl
However, there seems to be some hidden state that I don't understand, because after this command has been run once, it seems as if the --project option can be omitted. On the other hand, I have also tried to provide a nonsense project directory, e.g. /tmp:
julia --project=/tmp test/test_blueprint.jl
and sometimes it will still run the unit tests (!) and sometimes it won't. But when it fails to run the unit tests, it will succeed again as soon as I specify the correct path, that is /home/jonas/prog/julia/blueprint. I don't understand also how this interacts with whether I use using blueprint or include('../src/blueprint.jl') but it seems as if, when I use using, it works only iff the --project path is set correctly. But I am still not sure.
Up to now in my Lisp adventures I've just been pasting functions as written in a code editor into the REPL to run them, but I now have a program of sufficient size to build on that it will be convenient to use (load "filename.lisp") for the first time in my workflow.
Must I start using packages and/or namespacing to achieve this?
I find that when I use load, as above, I get
** - Continuable Error
DEFUN/DEFMACRO(CLASS): #<PACKAGE CLOS> is locked
If you continue (by typing 'continue'): Ignore the lock and proceed
The following restarts are also available:
SKIP :R1 skip (DEFMACRO CLASS # ...)
RETRY :R2 retry (DEFMACRO CLASS # ...)
STOP :R3 stop loading file /Users/m/cl/ansi-cl/ch17-objects/177d-new-full.lisp
ABORT :R4 Abort main loop
My .lisp file contains a macro called class, so I understand the error, sort of.
The thing is, when I paste the contents of the file directly into the REPL, I get no such error.
What's causing the difference in behaviour?
Is it packages, namespaces or something else?
I can indeed just type continue, and the file will load, but I'd like to understand what is happening here; what's the cause of this "Continuable error", and how should I deal with it if at all?
Package locking in CLISP is explained in the manual.
Symbol class is an ANSI CL symbol, so it cannot name any user-defined entity, and thus your program is not conforming, as explained in 11.1.2.1.2 Constraints on the COMMON-LISP Package for Conforming Programs.
You should rename your macro.
The lack of the error in the REPL is a bug in homebrew clisp.
CLISP as distributed with ubuntu works correctly.
When I build CLISP from sources on Mac, it works correctly too.
I am attempting to use MyPy with modules that use ruamel.yaml and Mypy cannot find ruamel.yaml even though Python has no problem finding it. I am puzzled because I can't find a module called YAML.py or class called YAML either, even though these statements work in Python:
from ruamel.yaml import YAML
yaml = YAML()
x = yaml.load()
What do I need to do to get MyPy to recognize ruamel.yaml?
A workaround is to run without the incremental logic of mypy:
python -m mypy --no-incremental myfile.py
Background
There is a known issue in mypy, see here.
In summary:
Something is not working with the incremental logic of mypy when it is encountering ruamel.
When you run it once, all goes ok. This is the command:
python -m mypy myfile.py
Then, when you run it again, you get an error:
error: Skipping analyzing 'ruamel': found module but no type hints or library stubs [import]
Then, when you run it again, it all goes ok
etc.
You should not be looking for a file YAML.py. The YAML in
yaml = YAML()
is a class that is defined in ruamel/yaml/main.py and that gets imported into ruamel/yaml/__init__.py (both under site-packages). That is why you do:
from ruamel.yaml import YAML
(the alternative would be that there is a file yaml.py under the directory ruamel, but the loader/dumper is a bit too much to put in one file).
What might work if the above knowledge doesn't help you resolve things, is explicitly set the global flag mypy_path or the environment variable MYPYPATH. This has to include the directory in which the directory ruamel is located.
( I could not find it mentioned in the documentation, but from the source ( mypy/build.py:mypy_path() ) you can see that this is supposed to be a string that gets split on os.pathsep (which is the colon (:) on my Linux based system))
I have the same issue.
Even after setting MYPYPATH=./.venv/lib/python3.7/site-packages
A temporary 'solution' is ignoring the missing import exception
mypy --ignore-missing-imports
I'm running a small Julia program using PyPlot in Juliabox (IJulia Notebook) but it's errors out with an error mesg as listed below. I not sure if it's trying to use my machine's disk to write to but I have valid R+W access there. Basically I'm trying out the examples as mentioned here: https://www.juliabox.org/notebooks/tutorial/Plotting%20in%20Julia.ipynb#
LoadError: unlink: read-only file system (EROFS)
Pkg.add("PyPlot")
using PyPlot
for i = 1.0:300.0
for j = 1.0+i:250.0, k=1.0:10
plot(i+j, i*k/j, color="red", linewidth=1.0, linestyle="--")
i += 0.1
j += 0.05
k += 0.01
end
end
Error log:
INFO: Nothing to be done
INFO: Precompiling module PyPlot...
INFO: Recompiling stale cache file /opt/julia_packages/.julia/lib/v0.4/Compat.ji for module Compat.
ERROR: LoadError: unlink: read-only file system (EROFS)
in unlink at fs.jl:102
in rm at file.jl:59
in create_expr_cache at loading.jl:330
in recompile_stale at loading.jl:461
in _require_from_serialized at loading.jl:83
in _require_from_serialized at ./loading.jl:109
in require at ./loading.jl:219
in include at ./boot.jl:261
in include_from_node1 at ./loading.jl:304
[inlined code] from none:2
in anonymous at no file:0
in process_options at ./client.jl:257
in _start at ./client.jl:378
while loading /home/juser/.julia/v0.4/PyCall/src/PyCall.jl, in expression starting on line 26
ERROR: LoadError: Failed to precompile PyCall to /home/juser/.julia/lib/v0.4/PyCall.ji
in error at ./error.jl:21
in compilecache at loading.jl:384
in require at ./loading.jl:224
in include at ./boot.jl:261
in include_from_node1 at ./loading.jl:304
[inlined code] from none:2
in anonymous at no file:0
in process_options at ./client.jl:257
in _start at ./client.jl:378
while loading /home/juser/.julia/v0.4/PyPlot/src/PyPlot.jl, in expression starting on line 5
LoadError: Failed to precompile PyPlot to /home/juser/.julia/lib/v0.4/PyPlot.ji
while loading In[10], in expression starting on line 2
in error at ./error.jl:21
in compilecache at loading.jl:384
in require at ./loading.jl:250
If I use 0.3.12 version (IJulia Notebook), then it compiles and shows INFO: Nothing to be done but doesn't show anything as output (some graphics plot diagram etc).
Thanks to ali_m. Here's the main summary on what that post says.
The problem seems to be that JuliaBox ships some precompiled cache files in a read-only directory /opt/julia_packages/.julia/lib/v0.4. If at some point it detects that the cache is stale and tries to recompile it, it fails.
This needs to be fixed in Julia itself—it shouldn't try to delete cache files from a read-only directory when recompiling.
Issue link is https://github.com/JuliaLang/julia/issues/14368
To resolve it only in 0.4.2, one can use (this will remove the 3rd index value from Base.LOAD_CACHE_PATH array/set/tuple) to your .juliarc file on JuliaBox to remove the read-only cache directory from the search path. Or just run this manually before typing using Compat etc to rebuild the cache without using the read-only search path.
splice!(Base.LOAD_CACHE_PATH, 3)
A nice example of splice! function
(PS: A function in Julia which ends with ! with its names means, that function will not only do its work but also will change its arguments data/value).
# Remove elements from an array by index with splice!
arr = [3,4,5]
splice!(arr,2) # => 4 ; arr is now [3,5]
The suggested workaround works for 0.4.2 (using echo 'splice!(Base.LOAD_CACHE_PATH, 3)' > ~/.juliarc.jl to insert the line into juliarc) but apparently LOAD_CACHE_PATH isn't defined while launching Julia 0.3.12 so this would fail there.
Adding the following line in the same file fixed this issue (adding a condition to work when version in Julia is 0.4 or above). I didn't see this issue in the 0.5 development version so we are good there.
VERSION >= v"0.4" && splice!(Base.LOAD_CACHE_PATH, 3)
I am a new user of Ocaml-java (or Cafesterol) which compiles primtive Ocaml program to executable jar that is allowed run on JVM. However when I try to compile a test program into executable jar I got error info as follow:
>java -jar ~/ocaml-project/ocamljava-bin-1.4/bin/ocamljava.jar -standalone regexdna.ml -o regexdna.jar
File "regexdna.ml", line 1, characters 0-1:
Error: No implementations provided for the following modules:
Str referenced from regexdna.cmj
Unix referenced from regexdna.cmj
It seems module Str and Unix is missing from Ocaml-java. However, str.jar and unix.jar do exist under ~/ocaml-project/ocamljava-bin-1.4/lib/others/ when I install Ocaml-java, and within these jars we do have Str.class and Unix.class. (I suppose this directory is on the path of the standard library of Ocaml-java, so it should be included in default search path)
Can any Ocaml-java user tell me how Ocaml-java search for dependency libraries?
Quoting Xavier Clerc on this :
Well it should work, but you have to pass explicitly the referenced
library (just as in vanilla OCaml). Leading in your case to:
$ /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java -jar ~/opt/ocamljava-2.0-early-access9/lib/ocamljava.jar str.cmja regexdna.ml
Note that I am using the latest ocamljava preview.