I would like to get path to my sqlite DB from a config file. How could I do that in Go?
This is a code, which I wrote before:
database, _ := sql.Open("sqlite3", "C:\\Users\\username\\project\\source.db")
In this case my path is "hard coded" directly in code. I would like to set a variable, which takes a path from a config data.
My first suggestion is that you use a JSON file rather than a YAML file for configuration, since Go supports it natively; you don't need to use any external packages.
type DBConfig struct {
Path string `json:"path"`
}
func loadConfig(path string) (*DBConfig, error) {
data, err := ioutil.ReadFile(path)
if err != nil {
return nil, err
}
var conf DBConfig
err = json.Unmarshal(data, &conf)
if err != nil {
return nil, err
}
return &conf, nil
}
My second suggestion is that you pass in the path to this config file in as a flag. You can supply a flag when you run your application like this:
$ go build . -o MyApp
$ ./MyApp --config=path/to/config/file
Flags are very powerful and allow you to easily configure your applications without changing much code. Using flags in Golang is simple.
var configPath = flag.String("config", "", "Path to file containing app config")
Just make sure that you add flag.Parse to the top of your main function in order to access them.
Here's a full example.
Good luck!
Related
I am trying to retrieve the filename from this http writer for testing purposes.
On the server I have:
func servefile(w http.ResponseWriter, r *http.Request ) {
...
// file.Name() is randomized with os.CreateTemp(dir, temp+"*"+ext) above
w.Header().Set("Content-Disposition", "attachment; filename="+file.Name())
w.Header().Set("Content-Type", "application/octet-stream")
http.ServeFile(w, r, file.Name()+".xyz") // serve file to user to download
...
}
*I put .xyz as a place holder for this demonstration
I am testing this function programatically with Go and I want to access the filename to able to save it in a variable in the client code.
I have looked at this post How can I parse the Content-Disposition header to retrieve the filename property? , but I have not gotten it to work. I have no clue what the filename is on the client side so I don't know how to reference it specifically. I know my code on the server side works, because when I send a request (through the browser) to this endpoint/function, the "Downloads" popup shows the file download progress with the name of the file.
EDIT** This is the client code I am calling it from:
func TestGetFile(t *testing.T) {
...
cid := "some string"
// requestfile() creates, executes, and returns an httptest.ResponseRecorder to the requestFile endpoint
reqfileRespRecorder := requestfile()
// createTmpFile creates a new file out of the contents recieved in requestfile()
filePath := "/tmp/temp.xyz"
file := createTmpFile(reqfileRespRecorder , filePath)
// CreateWriter() - writes file contents to body of multipart.Writer
w, body := createWriter(file)
// Create request to postRecord endpoint
req, err := http.NewRequest("POST", "/PostRecord?CID="+cid, body)
check(err)
req.Header.Add("Content-Type", w.FormDataContentType())
// execute request to PostRecord endpoint. returns an httptest.ResponseRecorder
respRecorder := executeRequest(PostRecord, req)
disposition, params, err := mime.ParseMediaType(`Content-Disposition`)
...
}
Based on #BaytaDarrell's comment. It dawned on me that I could print out the responses. That helped me realize that I was trying to find the content-disposition after the wrong request/response. The linked post still didn't help, but I got my code working like this:
func TestGetFile(t *testing.T) {
...
cid := "some string"
// requestfile() creates, executes, and returns an httptest.ResponseRecorder to the requestFile endpoint
reqfileRespRecorder := requestfile()
disp := reqfileRespRecorder.Header().Get("Content-Disposition")
line := strings.Split(disp, "=")
filename := line[1]
fmt.Println("filename: ", filename)
// createTmpFile creates a new file out of the contents recieved in requestfile()
filePath := "/tmp/temp.xyz"
file := createTmpFile(reqfileRespRecorder , filePath)
// CreateWriter() - writes file contents to body of multipart.Writer
w, body := createWriter(file)
...
}
Their comment realized I should re-look at the httptest package documentation. Here I found the Header() function and that I can use it to look at the header with Get().
This line reqfileRespRecorder.Header().Get("Content-Disposition") returns attachment; filename=temp37bf73gd.xyz and to store the filename in a variable I split on =.
I have the following piece of code, that serves static files from a static files directory (staticFilesDir):
for _, prefix := range []string{"css", "img", "js", "static"} {
prefix = "/" + prefix + "/"
fs := http.FileServer(http.Dir(staticFilesDir + prefix))
r.PathPrefix(prefix).Handler(http.StripPrefix(prefix, fs))
}
This directory changes from time to time, and currently I always need to restart the server process to use the new value.
How can I reconfigure/reload the FileServer without restarting the whole process?
Additional complication to that: other handlers of the http server are executing long-running jobs (incl. child processes etc.), that I would like to keep untouched during this reload.
What is the standard solution of this quite typical task?
You can add a level of indirection in between:
type MyFileServer struct {
sync.RWMutex
http.FileServer
}
func (f *MyFileServer) SetDir(dir string) {
f.Lock()
defer f.Unlock()
f.FileServer=http.FileServer(dir)
}
func (f *MyFileServer) ServeHTTP(w http.ResponseWriter,req *http.Request) {
f.RLock()
defer f.RUnlock()
f.FileServer.ServeHTTP(w,req)
}
I am trying to write a simple web app that will listen for UDP packets.
But I can either only listen for UDP packets, or run the web app...
I am not familiar with GoLang, but here's the code I'm using to...
listen for UDP:
ServerConn, _ := net.ListenUDP("udp", &net.UDPAddr{IP:[]byte{#,#,#,#},Port:####,Zone:""})
defer ServerConn.Close()
buf := make([]byte, 1024)
for {
n, addr, _ := ServerConn.ReadFromUDP(buf)
fmt.Println("Received ", string(buf[0:n]), " from ", addr)
}
Server logic:
package main
We import 4 important libraries
1. “net/http” to access the core go http functionality
2. “fmt” for formatting our text
3. “html/template” a library that allows us to interact with our html file.
4. "time" - a library for working with date and time.
import (
"net/http"
"fmt"
"time"
"html/template"
)
//Create a struct that holds information to be displayed in our HTML file
type Welcome struct {
Name string
Time string
}
//Go application entrypoint
func main() {
//Instantiate a Welcome struct object and pass in some random information.
//We shall get the name of the user as a query parameter from the URL
welcome := Welcome{"Anonymous", time.Now().Format(time.Stamp)}
//We tell Go exactly where we can find our html file. We ask Go to parse the html file (Notice
// the relative path). We wrap it in a call to template.Must() which handles any errors and halts if there are fatal errors
templates := template.Must(template.ParseFiles("templates/welcome-template.html"))
//Our HTML comes with CSS that go needs to provide when we run the app. Here we tell go to create
// a handle that looks in the static directory, go then uses the "/static/" as a url that our
//html can refer to when looking for our css and other files.
http.Handle("/static/", //final url can be anything
http.StripPrefix("/static/",
http.FileServer(http.Dir("static")))) //Go looks in the relative "static" directory first using http.FileServer(), then matches it to a
//url of our choice as shown in http.Handle("/static/"). This url is what we need when referencing our css files
//once the server begins. Our html code would therefore be <link rel="stylesheet" href="/static/stylesheet/...">
//It is important to note the url in http.Handle can be whatever we like, so long as we are consistent.
//This method takes in the URL path "/" and a function that takes in a response writer, and a http request.
http.HandleFunc("/" , func(w http.ResponseWriter, r *http.Request) {
//Takes the name from the URL query e.g ?name=Martin, will set welcome.Name = Martin.
if name := r.FormValue("name"); name != "" {
welcome.Name = name;
}
//If errors show an internal server error message
//I also pass the welcome struct to the welcome-template.html file.
if err := templates.ExecuteTemplate(w, "welcome-template.html", welcome); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
})
//Start the web server, set the port to listen to 8080. Without a path it assumes localhost
//Print any errors from starting the webserver using fmt
fmt.Println("Listening");
fmt.Println(http.ListenAndServe(":8080", nil));
}
taken from(https://medium.com/google-cloud/building-a-go-web-app-from-scratch-to-deploying-on-google-cloud-part-1-building-a-simple-go-aee452a2e654)
I tried putting both of these extracts in 1 file, as well as running 2 files at the same time using
go run *.go
Any help would be appreciated!
You're going to need to start looking into goroutines - since you're asking to do two things concurrently. I suggest doing some reading into channels, goroutines, and concurrency in general :)
Over here the Firebase docs explain how you can retrieve a token required to make requests to the Remote Config Rest API.
It provides example code for Python, Java and Node.js. Because there is no code for Go, it sends me to the Google Client Library (for Go). You might be able to understand why I am getting lost there...
The examples use GoogleCredential in Java, ServiceAccountCredentials in Python and google.auth.JWT in Node.js. I was not able to find any of those here. I do not know why there are no clear naming conventions.
I have found
firebaseremoteconfig-gen.go: The code looks like it already implements what the Firebase documentation page tries to achieve "manually". Comparison: doc, package.
Help
Because the "Usage example" of the package ends strangely abrupt and is the opposite of extensive, I do not understand how to make use of it.
I would be helped if someone could tell me how I can use this:
firebaseremoteconfigService, err := firebaseremoteconfig.New(oauthHttpClient)
I could not figure out where I would get oauthHttpClient from. There is an oauth2 package in the repository, but there I face the same problem:
oauth2Service, err := oauth2.New(oauthHttpClient)
I need oauthHttpClient again, so this cannot be a solution.
http.Client could be anything, but I need to authenticate with a service-account.json file, like shown in the three example snippets here.
Tags explanation
I hope that someone has either had experience with integrating Firebase Remote Config with Go, someone knows how Google Client API authentication works or someone is good enough with Go to get how the usage works.
There are a couple of main ways of authenticating with the google APIs, they are documented here:
Link to docs
The ways documented are "3-legged OAuth", "Using API Keys" and finally "Service Accounts".
From the links that you've included in the question; you are looking at the Python / Java / Node examples of "Service Accounts".
Using Service Accounts in go
The oauthHttpClient that you are referring to, is an http client that will attach the authentication information to the requests automatically.
You can create one using this package:
https://godoc.org/golang.org/x/oauth2/google
The examples linked in other languages use a "service account json key file".
Using the method linked below, you can read that keyfile and create a jwt.Config struct that will give you access to the client that you need.
https://godoc.org/golang.org/x/oauth2/google#JWTConfigFromJSON
The go equivalent of the other language examples linked is;
data, err := ioutil.ReadFile("/path/to/your-project-key.json")
if err != nil {
log.Fatal(err)
}
conf, err := google.JWTConfigFromJSON(data, "https://www.googleapis.com/auth/firebase.remoteconfig")
if err != nil {
log.Fatal(err)
}
// Initiate an http.Client. The following GET request will be
// authorized and authenticated on the behalf of
// your service account.
client := conf.Client(oauth2.NoContext)
client.Get("...")
I just started using the same library (from an AppEngine Standard project). This is how I am creating the service client:
import (
"context"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"golang.org/x/oauth2/google"
fb "google.golang.org/api/firebaseremoteconfig/v1"
"google.golang.org/appengine"
"google.golang.org/appengine/log"
)
const (
// Name of our service account file
saFileName = "my-firebase-sa.json"
// OAuth scopes used for remote config API
scopeRemoteConfig = "https://www.googleapis.com/auth/firebase.remoteconfig"
)
func createFirebaseService(ctx context.Context) (*fb.Service, error) {
data, err := ioutil.ReadFile(saFileName)
if err != nil {
return nil, err
}
conf, err := google.JWTConfigFromJSON(data, scopeRemoteConfig)
if err != nil {
return nil, err
}
return fb.New(conf.Client(ctx))
}
And I call it as such:
func fetchConfig(ctx context.Context) (*fb.RemoteConfig, error) {
s, err := createFirebaseService(ctx)
if err != nil {
log.Errorf(ctx, "Failed to create firebase service: %v", err)
return nil, fmt.Errorf("Failed to initialize Firebase service")
}
projectID := "projects/" + appengine.AppID(ctx)
cfg, err := s.Projects.GetRemoteConfig(projectID).Do()
if err != nil {
log.Errorf(ctx, "Failed to call Firebase remote config API: %v", err)
return nil, err
}
return cfg, nil
}
The code is using the Project ID to form its path; after reading through the lib code I noticed it was missing /projects/ from that path; so I just prepended that to my project ID and it works ;-) At least until they fix that and my code stops working..
Hopefully this helps someone.
I'm writing a Go Application that serves files in two different directories.
My project structure:
PROJECT_DIR
PROJECT_DIR/config
PROJECT_DIR/src
PROJECT_DIR/client
PROJECT_DIR/client/node_modules
PROJECT_DIR/client/www
in my main go file i start the file server using the following code:
func main() {
log.Print("started web server...");
httpsPortStr := ":" + strconv.FormatUint(config.CfgIni.HttpsPort, 10)
log.Printf("starting https web server at port %v", config.CfgIni.HttpsPort)
http.Handle("/", http.FileServer(http.Dir("client/www")))
http.Handle("/node_modules",http.FileServer(http.Dir(("client/node_modules"))))
err := http.ListenAndServeTLS(httpsPortStr, config.CfgIni.CertificateFile, config.CfgIni.PrivateKeyFile, nil)
if err != nil {
log.Fatalf("https server stopped with the following error: %v", err)
} else {
log.Print("https server stopped with no error")
}
}
as you can see I mapped / to client/www and /node_modules to client/node_modules.
when I try to access files on client/www for example https://host:port/test.html, it works great!
when I try to access files on client/node_modules for example: https://host:port/node_modules/test.html, I get 404 page not found.
test.html file exists in both location and is readable (no permission problems).
I'm probably configuring the routing wrong somehow.
any ideas?
thanks!
The FileServer is trying to route paths such as /node_modules/... to the file "client/node_modules/node_modules/..."
So use StripPrefix, eg:
http.Handle("/node_modules", http.StripPrefix("/node_modules", http.FileServer(http.Dir(("client/node_modules")))))
See another answer here.