I have a dotnet core gRPC project and I'm trying to include route annotations in my proto files like below:
import "google/api/annotations.proto";
file structure is like this (for the reason being that I imported googleapis repository as a git submodule):
protos/
myproto.proto
googleapis/
google/
api/
annotations.proto
...
in a go project it can be done by:
protoc -I . -I ./googleapis --go_out=plugins=grpc:. *.proto
where -I ./googleapis gives compiler the dir where it can find annotations.proto file and its dependencies.
But when I'm using MSBuild in a dotnet grpc project using config like below, I could not figure out how to include custom directories.
<ItemGroup>
<Protobuf Include="protos/*.proto" GrpcServices="Server" />
</ItemGroup>
Official doc mentioned an alternative to customize target build so that I can use protoc:
protoc --plugin=protoc-gen-grpc=$(gRPC_PluginFullPath) -I $(Protobuf_StandardImportsPath) ...
but above command ignores service definition and does not generate server stub code as mentioned here, while MSBuild does.
A workaround I found but not ideal:
I realize Grpc.Tools dotnet package distributes some commonly used proto files, so I copied annotations.proto and its dependencies there (in macOS) and it worked:
`~\.nuget\packages\grpc.tools\2.25.0\build\native\include`
Updates:
Another workaround:
The project root directory is included by default, so use it as the base path and copy the imported proto files there also works (better but still not ideal).
Any ideas how to include custom directories like above through MSBuild?
Finally figured out. As Jan Tattermusch suggested, ProtoRoot specifies the directories from which msbuild Include property as well as the proto import keyword will be looking for files, whereas the way ProtoRoot works for Include is different from how it works for import. Thus in order to customize the proto file structures to be like this, ProtoRoot must include all different paths which should be:
<Protobuf Include="protos/*.proto" ProtoRoot="./protos; ./protos/googleapis" ... />
Updates:
Above works fine for grpc-dotnet version prior to v2.31.0. For newer versions, warnings will show up saying ProtoRoot path is not a prefix of Include path. To fix it, use below config instead:
<Protobuf Include="protos/*.proto" ProtoRoot="protos" AdditionalImportDirs="protos/googleapis" ... />
Related
I need to create a self-contained .net core (3.1 in my case) app and to pack & publish using chocolatey so it can be installed and used.
I'm using Azure DevOps and have a feed on my own where I'm supposed to publish the chocolatey package.
The objective is to do this in the build pipeline, so, among other tasks I have:
dotnet publish task: creates the self-contained executable
chocolatey pack task: creates the .nupkg from a very simplistic .nuspec (only mandatory fields) I created.
My current problem is that the .nupkg file created contains always the project files and not the executable generated.
To try and work around it I even made the chocolatey pack's task work directory the same as the dotnet publish's task output one.
What am I missing? Is there another approach?
Azure DevOps: publish self-contained .net Core app with Chocolatey
It depends on whether you include the contained executable file in your .nuspec file.
If we include the contained executable file in the .nuspec file, chocolatey will create the .nupkg include the contained executable file, like:
<files>
<file src="IngestCanonicLtesConsole\ContainedExecutable.exe" target="Tools\ContainedExecutable.exe" />
</files>
We could add this contained executable file in the package:
So, if we are only include the mandatory fields without the <files>contained executable </files>, it will not include the contained executable file.
Besides, we need to include the contained executable file in the .nuspec file, we could change the output of the dotnet build to $(System.DefaultWorkingDirectory)\IngestCanonicLtesConsole, so that we could use the relative path in the .nuspec file.
Please check the document .nuspec reference for some more details.
After a few tests I realized that the chocolatey pack will "pack" all files that in exist in the same folder as the ".nuspec". Not sure this is because I don't set anything on tool.
Basically, my solution was to copy my ".nuspec" file to the folder where my executable was.
I tried using Dotpeek and ILSpy.Net to decompile (my own code), they failed.
Do I need special obfuscation on distributed binaries of .Net Core 3 self-contained single executable ?
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp3.0</TargetFramework>
<PublishTrimmed>true</PublishTrimmed>
<PublishReadyToRun>true</PublishReadyToRun>
<PublishSingleFile>true</PublishSingleFile>
<RuntimeIdentifier>win-x64</RuntimeIdentifier>
</PropertyGroup>
</Project>
The single-file exe is really an unmanaged wrapper and ILSpy doesn't support decompiling this. But when you run the exe, it unwraps its contents to a temp folder. So you can find the managed dll there and decompile it using ILSpy.
To find the temp folder, you can use any tool that shows locations of assemblies loaded by a process. SysInternals Process Monitor (procmon) is a good one.
You can setup procmon to filter by your exe name, and when you launch your exe, procmon should show some events for assemblies being loaded from a temp folder:
You can browse to that folder and find your managed dll there. And you can decompile using ILSpy from that location.
I wrote a blog entry: https://eersonmez.blogspot.com/2020/02/ilspy-decompiling-net-core-self.html
I wrote a small dotnet tool after I stumbled upon this question and couldn't find a lightweight tool myself other than ILSpy.
You can install it using the following dotnet command: dotnet tool install -g sfextract.
After installing it, run it using the following command: sfextract application.exe -o output-dir
The bundle format for .NET 5.0 (bundle version 2) is identical to previous versions. .NET 6.0 (bundle version 6) has an additional field for each file entry containing the compressed size, since single-file applications can now be compressed using gzip by setting EnableCompressionInSingleFile to true.
https://www.nuget.org/packages/sfextract/
https://github.com/Droppers/SingleFileExtractor
Update 07/2022: .Net 5 single-file does not automatically unpack to the same temporary location as before. to force it to be unpacked you would need to add the following:
in the project file add these properties (according to theseMicrosoft docs):
<PublishSingleFile>true</PublishSingleFile>
<IncludeAllContentForSelfExtract>true</IncludeAllContentForSelfExtract>
Add an environment variable DOTNET_BUNDLE_EXTRACT_BASE_DIR with the location you want the files extracted to.
Update: One of the announcements made regarding .Net 5 states that the way single-file executables will be made would change, so this method will not work for them.
I wanted to add on #Eren Ersönmez's answer, that while ILSpy DotPeek don't support this at the time, since the self-contained single file is just a wrapper that contains all your DLLs and gets extracted on runtime, simply knowing where it is extracted to can save you using ProcMon or ProExp or windbg.
If you use windows you can go to c:\Users\{Local Username}\AppData\local\temp\.net\{Name of executable}
which should lead to somewhere similar to
c:\Users\alenros\AppData\Local\Temp.net\MyTestApplication
Launch your exe, and a folder with the same name will be created in that location.
The folder will contain randomly named folders. open the latest one and there you will find all your extracted DLLs, which can then be decompiled.
The dotnet publish command published into the projects bin/netcoreapp2.2/Debug/publish folder. Where netcoreapp2.2 presumably changes with the dotnet version and Debug changes with whatever configuration the -c parameter specifies.
For CI/CD purposes this is clearly undesirable. Alternatively one can pass -o to pass an explicit output path, but again, in a CI/CI environment this path should be inside the project folder structure, e.g. something like:
dotnet publish -o publish
But, because the publish command globs up all files, it picks up previous publish attempts and recursively stores them. This can be mitigated by either cleaning the publish folder explicitly, and/or adding a to the csproj for the project but now there is a dependency between the build script and the csproj: if the publish path is changed in the build scripts for any reason without a corresponding csproj update things break.
So, the least fragile option seems to be to use the default output path as thats automatically excluded from globbing, but how to remove the version & configuration sensitivity? Is there a particularly safe way to get dotnet to tell my CI/CD environment what its output path for build / publish is?
IMP: I do not have enough so reputation to add comment
refer : dotnet publish
you could use relative path with -o option and you may end up avoiding folder name with run-time, platform identification explicitly.
or why do not you consider using build command with publish profile where you can specify explicit path. but generally relative path is less error prone.
Hope this may help you !
I'm working on an SBT project that has to be built with the options like:
-Xmx2G -Xss256M -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
This means that every new developer has to read the readme and assign the options to SBT_OPTS in bash profile or put them in the sbtopts file. Similarly, this has to be configured on Jenkins and this applies to all the projects (so if someone wants to use -XX:+UseG1GC with other projects it becomes an issue). Is it possible to specify the required options in the build file itself? This seems logical to me, as the options are project-specific and without them, you cannot build the project.
Create a .sbtopts file at the root of the build with contents:
-J-Xmx2G
-J-Xss256M
-J-XX:+UseConcMarkSweepGC
-J-XX:+CMSClassUnloadingEnabled
I created simple Java Servlet: WelcomeServlet.java.
Than, I tried compile this file via:
javac WelcomeServlet.java
In result I see compile error:
package javax.servlet doesn't exit
I try find solution for this error with Google. And I find first part of answer: java compiler doesnt see servlet-api.jar file.
I know, that Apache Tomcat in it lib folder contains servlet-api.jar file.
So, I have this file, but where I must copy this file??
I try different folders:
echo %JAVA_HOME%
C:\Program Files\Java\jdk1.6.0_26
%PATH% contains this line: C:\Program Files\Java\jdk1.6.0_26\bin
So, I copy in:
%JAVA_HOME%\bin
%JAVA_HOME%\lib
%JAVA_HOME%\jre\lib
And in result same error.
And only after I copy servlet-api.jar in directory:
%JAVA_HOME%\jre\lib\ext
compilation complite sucessful.
My question: Why? Why I must copy in folder %JAVA_HOME%\jre\lib\ext ??
Where This moment describe in documentation?
And other question we have some official docs or specifications that describe folder structure for jdk folder??
You'll need to specify the directory or directories you want the compiler to search by using the -classpath command line option when running javac. The reason the compiler found your .jar in %JAVA_HOME%\jre\lib\ext is because it searches the extension directories by default.
This is for Java 1.5, but I believe it is more or less still correct:
http://docs.oracle.com/javase/1.5.0/docs/tooldocs/findingclasses.html
The link Shaun provides is a more complete answer. But in short, using the classpath is the best way to introduce 3rd party or external (to the JDK/JRE) libraries. The classpath is a concept much like the %PATH% or the $PATH variables, but specifies locations for java to use for lookup rather than the shell to use for lookup of executables.
The classpath provides the java compiler or java virtual machine a list of items to use when searching for resources. This "path" may include directories or files. It will typically include jar files and sometimes locations of configuration files. Many Java based lookup schemes for files configuration or otherwise use some variant of what is accomplished by [Class#getResourceAsStream()][1]'s use of walking the Classpath.
I have rarely seen an incident where putting a jar file in the lib/ext location was preferred to utilizing the Classpath.
The classpath is typically an environment variable (%CLASSPATH% or $CLASSPATH) or specified on the command line when running java or javac (e.g. -cp or -classpath see the help from the executable you are running).
Build tools such as Ant and Maven will also provide abstractions to defining the list of jars to be utilized by your applications and are highly recommended to be used for any length of repetitive change code, build, test, run cycles.