I'm using gin framework. And I'm opening the sqlite database in the main function like this
func main() {
...
db, err := sql.Open("sqlite3", "./libreread.db")
CheckError(err)
defer db.Close()
...
}
And I have these router handlers in the main function.
...
r.GET("/", GetHomePage)
r.GET("/signin", GetSignIn)
r.POST("/signin", PostSignIn)
...
How to pass that db value through the router handler func PostSignin(c *gin.Context) ?
So that I could avoid opening and closing the database each time in the functions.
UPDATE: I'm using go-sqlite3 package.
Thanks!
Let's say you have your sql client initialized in db, then, you can pass it to different routes with
r.GET("/", GetHomePageHandler(&db))
And in your GetHomePageHandler:
func GetHomePageHandler(sqldb *SQLiteConn) func (*gin.Context) {
return func (*gin.Context) {
. . .
}
}
Where *SQLiteConn is the type of your sql db instance. I don't know which package you are currently using so this is just an example.
You can find also a more elegant way of solving it in this answer,
Related
Is there any way to prevent httputil.ReverseProxy from sending an incoming request to the target server? For example, if I have a cache and I can respond to the client using only local data. Or after validation, I want to return an error to the client.
A httputil.ReverseProxy has a single exported method, ServeHTTP(rw http.ResponseWriter, req *http.Request) which makes it implement the net/http.Handler interface.
So basically at a place you're now using an vanilla httputil.ReverseProxy instance, instead use an instance of your custom type which implements net/http.Handler as well, keeps a pointer to an instance of httputil.ReverseProxy, and either processes the request itself or calls out to that ReverseProxy instance's ServeHTTP.
You should be able to wrap the http.DefaultTransport with a cache that can either use the cache based on the request or fallback on the http.DefaultTransport.
package main
import (
"net/http"
"net/http/httputil"
)
var _ http.RoundTripper = &CachingTransport{}
type CachingTransport struct {
// put your cache here
}
func (c *CachingTransport) RoundTrip(request *http.Request) (*http.Response, error) {
// determine whether to use the cache and return, or use the default transport
return http.DefaultTransport.RoundTrip(request)
}
func main() {
_ = httputil.ReverseProxy{
Transport: &CachingTransport{},
}
}
This question already has answers here:
How to mock http.Client Do method
(4 answers)
Closed 4 months ago.
Hey so I have seen and used this post to help mock my http.Client but when I try to pass the mock request I get the following error: cannot use mockClient (variable of type *MockClient) as *"net/http".Client value in argument to api.callAPI.
In one file I have my actual code:
I created the HTTPClient:
type HTTPClient interface {
Do(req *http.Request) (*http.Response, error)
}
I have the function that passes the HTTPClient as an interface (I can't show all of it because it is for work but here's the important pieces):
func (api *API) callAPI(req *http.Request, client HTTPClient) (utils.ErrorWrapper, bool) {
response, err := client.Do(req)
}
I also have another function that calls the callAPI method. In that function I create client variable right before I call the callAPI function
var Client HTTPClient = &http.Client{}
response, isRetry := api.callAPI(req, Client)
This all works fine. However, in my testing file I get the error as mentioned above. I am using testify for my mocking framework. Here is what I have in my testing file (both the testing file and the actual code are apart of the same package):
set up my mock client and the Do function using testify
type MockClient struct {
mock.Mock
}
func (m *MockClient) Do(req *http.Request) (*http.Response, error) {
args := m.Called()
resp := args.Get(0)
return resp.(*http.Response), args.Error(1)
}
Then create my test:
func TestCallAPI(t *testing.T) {
mockClient := &MockClient{}
recorder := httptest.NewRecorder()
responseCh := make(chan utils.ErrorWrapper)
c, _ := gin.CreateTestContext(recorder)
id:= "unitTest123"
api := NewAPICaller(responseCh, id, c)
var response = Response{
StatusCode: 200,
}
//setup expectations
mockClient.On("Do").Return(response, nil)
req, _ := http.NewRequest("GET", "URL I Can't Show", nil)
wrapper, isRetry := api.callAPI(req, mockClient)
mockClient.AssertExpectations(t)
assert.Equal(t, "placeholder", wrapper)
assert.Equal(t, false, isRetry)
}
I tried to do a similar thing with the mockclient variable the way I did with the Client variable:
var mockclient HTTPClient = &MockClient{}
but I get this error on the HTTPClient: undeclared name: HTTPClient. Unsure why this is happening because they are a part of the same package so I thought it could be exported easily?
There are 2 ways you can achieve this.
As suggested by #Volker. Setup a HTTP Server. you can use a docker container to run it for your tests.
The other options (my favorite for unit testing) is to wrap your HTTP client in an interface of your own. That interface will expose methods either your domain specific OR probably just plain simple GET, POST. Then use the library like mockgen to generate mock. This library generate mocks based on interface. With this setup in place, you can have proper unit tests on the exposed methods.
Are there any options for logging request times in grpc-node? I've been able to log response times using opentelemetry & jaegar (to display response times). I couldn't find any npm packages for this either, but just wanted to ask you guys if you did find any options for grpc-node.
You don't need a package to do it, you can do it using a simple gRPC client interceptor.
This is how you would do it in Golang. Just check how you can create a gRPC interceptor in Node. Sorry for not having a JS example, hope it helps some how.
func UnaryRequestTImeInterceptor() grpc.UnaryClientInterceptor {
return func(
ctx context.Context,
method string,
req interface{},
reply interface{},
cc *grpc.ClientConn,
invoker grpc.UnaryInvoker,
opts ...grpc.CallOption,
) error {
start := time.Now()
err := invoker(ctx, method, req, reply, cc, opts...)
reqTime := time.Since(start)
return err
}
}
Over here the Firebase docs explain how you can retrieve a token required to make requests to the Remote Config Rest API.
It provides example code for Python, Java and Node.js. Because there is no code for Go, it sends me to the Google Client Library (for Go). You might be able to understand why I am getting lost there...
The examples use GoogleCredential in Java, ServiceAccountCredentials in Python and google.auth.JWT in Node.js. I was not able to find any of those here. I do not know why there are no clear naming conventions.
I have found
firebaseremoteconfig-gen.go: The code looks like it already implements what the Firebase documentation page tries to achieve "manually". Comparison: doc, package.
Help
Because the "Usage example" of the package ends strangely abrupt and is the opposite of extensive, I do not understand how to make use of it.
I would be helped if someone could tell me how I can use this:
firebaseremoteconfigService, err := firebaseremoteconfig.New(oauthHttpClient)
I could not figure out where I would get oauthHttpClient from. There is an oauth2 package in the repository, but there I face the same problem:
oauth2Service, err := oauth2.New(oauthHttpClient)
I need oauthHttpClient again, so this cannot be a solution.
http.Client could be anything, but I need to authenticate with a service-account.json file, like shown in the three example snippets here.
Tags explanation
I hope that someone has either had experience with integrating Firebase Remote Config with Go, someone knows how Google Client API authentication works or someone is good enough with Go to get how the usage works.
There are a couple of main ways of authenticating with the google APIs, they are documented here:
Link to docs
The ways documented are "3-legged OAuth", "Using API Keys" and finally "Service Accounts".
From the links that you've included in the question; you are looking at the Python / Java / Node examples of "Service Accounts".
Using Service Accounts in go
The oauthHttpClient that you are referring to, is an http client that will attach the authentication information to the requests automatically.
You can create one using this package:
https://godoc.org/golang.org/x/oauth2/google
The examples linked in other languages use a "service account json key file".
Using the method linked below, you can read that keyfile and create a jwt.Config struct that will give you access to the client that you need.
https://godoc.org/golang.org/x/oauth2/google#JWTConfigFromJSON
The go equivalent of the other language examples linked is;
data, err := ioutil.ReadFile("/path/to/your-project-key.json")
if err != nil {
log.Fatal(err)
}
conf, err := google.JWTConfigFromJSON(data, "https://www.googleapis.com/auth/firebase.remoteconfig")
if err != nil {
log.Fatal(err)
}
// Initiate an http.Client. The following GET request will be
// authorized and authenticated on the behalf of
// your service account.
client := conf.Client(oauth2.NoContext)
client.Get("...")
I just started using the same library (from an AppEngine Standard project). This is how I am creating the service client:
import (
"context"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"golang.org/x/oauth2/google"
fb "google.golang.org/api/firebaseremoteconfig/v1"
"google.golang.org/appengine"
"google.golang.org/appengine/log"
)
const (
// Name of our service account file
saFileName = "my-firebase-sa.json"
// OAuth scopes used for remote config API
scopeRemoteConfig = "https://www.googleapis.com/auth/firebase.remoteconfig"
)
func createFirebaseService(ctx context.Context) (*fb.Service, error) {
data, err := ioutil.ReadFile(saFileName)
if err != nil {
return nil, err
}
conf, err := google.JWTConfigFromJSON(data, scopeRemoteConfig)
if err != nil {
return nil, err
}
return fb.New(conf.Client(ctx))
}
And I call it as such:
func fetchConfig(ctx context.Context) (*fb.RemoteConfig, error) {
s, err := createFirebaseService(ctx)
if err != nil {
log.Errorf(ctx, "Failed to create firebase service: %v", err)
return nil, fmt.Errorf("Failed to initialize Firebase service")
}
projectID := "projects/" + appengine.AppID(ctx)
cfg, err := s.Projects.GetRemoteConfig(projectID).Do()
if err != nil {
log.Errorf(ctx, "Failed to call Firebase remote config API: %v", err)
return nil, err
}
return cfg, nil
}
The code is using the Project ID to form its path; after reading through the lib code I noticed it was missing /projects/ from that path; so I just prepended that to my project ID and it works ;-) At least until they fix that and my code stops working..
Hopefully this helps someone.
In Go, a TCP connection (net.Conn) is a io.ReadWriteCloser. I'd like to test my network code by simulating a TCP connection. There are two requirements that I have:
the data to be read is stored in a string
whenever data is written, I'd like it to be stored in some kind of buffer which I can access later
Is there a data structure for this, or an easy way to make one?
No idea if this existed when the question was asked, but you probably want net.Pipe() which provides you with two full duplex net.Conn instances which are linked to each other
EDIT: I've rolled this answer into a package which makes things a bit simpler - see here: https://github.com/jordwest/mock-conn
While Ivan's solution will work for simple cases, keep in mind that a real TCP connection is actually two buffers, or rather pipes. For example:
Server | Client
---------+---------
reads <=== writes
writes ===> reads
If you use a single buffer that the server both reads from and writes to, you could end up with the server talking to itself.
Here's a solution that allows you to pass a MockConn type as a ReadWriteCloser to the server. The Read, Write and Close functions simply proxy through to the functions on the server's end of the pipes.
type MockConn struct {
ServerReader *io.PipeReader
ServerWriter *io.PipeWriter
ClientReader *io.PipeReader
ClientWriter *io.PipeWriter
}
func (c MockConn) Close() error {
if err := c.ServerWriter.Close(); err != nil {
return err
}
if err := c.ServerReader.Close(); err != nil {
return err
}
return nil
}
func (c MockConn) Read(data []byte) (n int, err error) { return c.ServerReader.Read(data) }
func (c MockConn) Write(data []byte) (n int, err error) { return c.ServerWriter.Write(data) }
func NewMockConn() MockConn {
serverRead, clientWrite := io.Pipe()
clientRead, serverWrite := io.Pipe()
return MockConn{
ServerReader: serverRead,
ServerWriter: serverWrite,
ClientReader: clientRead,
ClientWriter: clientWrite,
}
}
When mocking a 'server' connection, simply pass the MockConn in place of where you would use the net.Conn (this obviously implements the ReadWriteCloser interface only, you could easily add dummy methods for LocalAddr() etc if you need to support the full net.Conn interface)
In your tests you can act as the client by reading and writing to the ClientReader and ClientWriter fields as needed:
func TestTalkToServer(t *testing.T) {
/*
* Assumes that NewMockConn has already been called and
* the server is waiting for incoming data
*/
// Send a message to the server
fmt.Fprintf(mockConn.ClientWriter, "Hello from client!\n")
// Wait for the response from the server
rd := bufio.NewReader(mockConn.ClientReader)
line, err := rd.ReadString('\n')
if line != "Hello from server!" {
t.Errorf("Server response not as expected: %s\n", line)
}
}
Why not using bytes.Buffer? It's an io.ReadWriter and has a String method to get the stored data. If you need to make it an io.ReadWriteCloser, you could define you own type:
type CloseableBuffer struct {
bytes.Buffer
}
and define a Close method:
func (b *CloseableBuffer) Close() error {
return nil
}
In majority of the cases you do not need to mock net.Conn.
You only have to mock stuff that will add time to your tests, prevent tests from running in parallel (using shared resources like the hardcoded file name) or can lead to outages (you can potentially exhaust the connection limit or ports but in most of the cases it is not a concern, when you run your tests in isolation).
Not mocking has an advantage of more precise testing of what you want to test with a real thing.
https://www.accenture.com/us-en/blogs/software-engineering-blog/to-mock-or-not-to-mock-is-that-even-a-question
Instead of mocking net.Conn, you can write a mock server, run it in a goroutine in your test and connect to it using real net.Conn
A quick and dirty example:
port := someRandomPort()
srv := &http.Server{Addr: port}
go func(msg string) {
http.HandleFunc("/hello", myHandleFUnc)
srv.ListenAndServe()
}
myTestCodeUsingConn(port)
srv.Shutdown(context.TODO())