I am trying to download something from 3 servers. My idea is if the first server is closed,it will use the second server. I noticed,if the first server has been closed, it will created a run time error.I want to know how to use this error,what i need is like this:
if run time err!=nil{do something}
i am new to golang,hope someone can help me
thank you
To elaborate on what FUZxxl explained, go makes a distinction between an error (something which could go wrong indeed went wrong) and an exception (something which could not possibly go wrong actually went wrong).
The distinction can sometimes be subtle (as it relies on what is 'unexpected'), but it can also be clearer than the 'everything is an exception' that you see in other languages.
For instance, consider integers which might overflow. One possibility is to consider it a 'normal' behaviour, which should be handled appropriately:
func safe_add(x, y uint32) (uint32, error) {
z := x + y
if z < x || z < y {
return 0, fmt.Errorf("Integer overflow")
}
return z, nil
}
Another is to consider it 'never happens' and have the runtime panic in the unlikely case when it happens against all odds:
func panic_add(x, y uint32) uint32 {
z, err := safe_add(x, y)
if err != nil {
panic(err)
}
return z
}
(Note that I use my own 'safe_add' here, but you don't have to of course)
The main difference is in the way you handle the error afterwards. Adding a number to itself until it overflows with errors gives:
func safeloop(u uint32) {
var err error
for {
if u, err = safe_add(u, u); err != nil {
fmt.Println(err)
return
} else {
fmt.Println(u)
}
}
}
While handling panics uses the recover built-in function:
func panicloop(u uint32) {
defer func() {
if err := recover(); err != nil {
fmt.Println(err)
}
}()
for {
u = panic_add(u, u)
fmt.Println(u)
}
}
(full examples on the playground)
Note that the panic version has a much simpler loop, as you basically never expect anything to go wrong and never check for errors. The counterpart for this is that the way to handle panics is quite cumbersome, even for a very simple example like this. You defer a function which will call recover and capture the error when it arises and breaks out of the function. When your code becomes more complex, tracking exactly where/how the panic arose and acting on it accordingly can become much more complex than checking for errors in places where they could arise, with the result, err := func_which_may_fail(...) pattern.
You can even alternate between panics, recover which return errors, errors converted to panics, ... but this is (understandably) considered poor design.
There are some good resources on error handling and panics on the go blog. The specs is a good read to.
In your case, as you expect 'the server is closed' to be a pretty frequent behaviour, you should definitely go the error way, as FUZxxl suggested, but I hope this might be useful to you (or others) to understand how error handling works in Go.
When you do something which could go wrong, you get an error object.
bytes, err = stream.Read(buffer)
To check whether what you tried actually went wrong, compare the error object against nil. nil signalizes that no error has happened. In the case the error is not nil, you can do something about it.
if err != nil {
// insert error handling here
}
Related
I am very new to Go and a beginner to programming. I cam across this:
res, err := http.Get("URL_HERE")
if err != nil {
log.Fatal(err)
}
Can anyone please help me understand this? Does http.Get return two values? Is err a pointer?
Yes, http.Get returns two values. The documentation of http.Get is here, and this is its signature:
func Get(url string) (resp *Response, err error)
err is a value of the error type, which is an interface so it could be nil. From its documentation:
The error built-in interface type is the conventional interface for
representing an error condition, with the nil value representing no
error.
I strongly suggest you start learning Go by completing the online tour, and then check out additional learning resources at https://learn.go.dev/
Now i have a map with only one write/delete goroutine and many read goroutines, there are some solutions upon Map with concurrent access, such as RWMutex, sync.map, concurrent-map, sync.atomic, sync.Value, what's the best choice for me?
RWMutex's read lock is a little redundant
sync.map and concurrent-map focus on many write goroutines
Your question is a little vague - so I'll break it down.
What form of concurrent access should I use for a map?
The choice depends on the performance you require from the map. I would opt for a simple mutex (or a RWMutex) based approach.
Granted, you can get better performance from a concurrent map. sync.Mutex locks all of a maps buckets, whereas in a concurrent map, each bucket has it's own sync.Mutex.
Again - it all depends on the scale of your program and the performance you require.
How would I use a mutex for concurrent access?
To ensure the map is being used correctly, you can wrap this in a struct.
type Store struct {
Data map[T]T
}
This a more object-oriented solution, but it allows us to make sure any read/writes are performed concurrently. As well as this, we can easily store other information that may be useful for debugging or security, such as author.
Now, we would implement this with a set of methods like so:
mux sync.Mutex
// New initialises a Store type with an empty map
func New(t, h uint) *Store {
return &Store{
Data: map[T]T{},
}
}
// Insert adds a new key i to the store and places the value of x at this location
// If there is an error, this is returned - if not, this is nil
func (s *Store) Insert(i, x T) error {
mux.Lock()
defer mux.Unlock()
_, ok := s.Data[i]
if ok {
return fmt.Errorf("index %s already exists; use update", i)
}
s.Data[i] = x
return nil
}
// Update changes the value found at key i to x
// If there is an error, this is returned - if not, this is nil
func (s *Store) Update(i, x T) error {
mux.Lock()
defer mux.Unlock()
_, ok := s.Data[i]
if !ok {
return fmt.Errorf("value at index %s does not exist; use insert", i)
}
s.Data[i] = x
return nil
}
// Fetch returns the value found at index i in the store
// If there is an error, this is returned - if not, this is nil
func (s *Store) Fetch(i T) (T, error) {
mux.Lock()
defer mux.Unlock()
v, ok := s.Data[i]
if !ok {
return "", fmt.Errorf("no value for key %s exists", i)
}
return v, nil
}
// Delete removes the index i from store
// If there is an error, this is returned - if not, this is nil
func (s *Store) Delete(i T) (T, error) {
mux.Lock()
defer mux.Unlock()
v, ok := s.Data[i]
if !ok {
return "", fmt.Errorf("index %s already empty", i)
}
delete(s.Data, i)
return v, nil
}
In my solution, I've used a simple sync.Mutex - but you can simply change this code to accommodate RWMutex.
I recommend you take a look at How to use RWMutex in Golang?.
I'm a Golang newbie but I thought I had got the essentials of pointers and references straight, but apparently not:
I have a method that must return a []github.Repository, which is a type from the Github client in go.
The API call returns the results paginated so I must cycle until there's no more results, and add the result of each call to the allRepos variable, and return that. Here's what I have so far:
func (s *inmemService) GetWatchedRepos(ctx context.Context, username string) ([]github.Repository, error) {
s.mtx.RLock()
defer s.mtx.RUnlock()
opt := &github.ListOptions{PerPage: 20}
var allRepos []github.Repository
for {
// repos is of type *[]github.Repository
repos, resp, err := s.ghClient.Activity.ListWatched(ctx, "", opt)
if err != nil {
return []github.Repository{}, err
}
// ERROR: Cannot use repos (type []*github.Repository) as type github.Repository
// but dereferencing it doesn't work, either
allRepos = append(allRepos, repos...)
if resp.NextPage == 0 {
break
}
opt.Page = resp.NextPage
}
return allRepos, nil
}
My question: how can I append the results of each call and return a result of type []github.Repository?
Also, why doesn't dereferencing work here? I've tried replacing allRepos = append(allRepos, repos...) with allRepos = append(allRepos, *(repos)...) but I get this error message:
Invalid indirect of (repos) (type []*github.Repository)
Well, something is not okay here:
You say in the comment that "repos is of type *[]github.Repository" but the compiler's error message indicates that repos is of type []*Repository". The compiler is never (except when buggy) wrong.
Note that *[]github.Repository and []*Repository are completely different types, especially the second is not a slice of Repositories and you cannot (really, there is no way) dereference these pointers during append(): You have to write a loop and dereference each slice item and append one by one.
What is strange too: github.Repository and Repository seem to be two different types one from package github, the other from the current package. Again, you'll have to get that straight too.
Note that there are no references in Go. Stop thinking about these immediately: This is a concept from other languages which is not helpful (as inexistent) in Go.
In your example the dereferencing is not correct. You should make it like this:
allRepos = append(allRepos, *repos...)
Here a simple example with dereferencing a pointer to a slice of string. https://play.golang.org/p/UDzaG5z8Pf
Say I have this struct:
type Foo struct {
Bar *string `json:"bar"`
Baz *int64 `json:"baz,omitempty"`
Qux *string `json:"qux"`
Quux string `json:"quux"`
}
After unmarshalling the json, I check for nil like so:
switch {
case f.Bar == nil:
return errors.New("Missing 'bar'")
case f.Baz == nil:
f.Baz = 42
case f.Qux == nil:
return errors.New("Missing 'qux'")
}
(or through a series of if statements, etc...)
I understand that I can put all the nil comparisons in one comma separated case, but each nil check will have differing returns.
My question: is there a less verbose way of doing the nil checks?
A question to you: how less verbose you want to get? Because you want to do different things on different conditions (different fields being nil). Your code contains these different things and the different conditions. Beyond that what's "redundant" in your code are just the switch and case keywords. You want to leave them out? Because the rest is not "redundant", they are required.
Also note that in Go cases do not fall through even without a break (unlike in other languages), so in your above example if f.Baz is nil, you will set it to 42 and f.Qux will not be checked (so no error will be returned), but if f.Baz is non-nil and f.Qux is nil, an error will be returned. I know it's just an example, but this is something to keep in mind. You should handle errors first if you use a switch! Or use if statements and then the error will be detected and returned regardless of the order of field checks.
Your code with switch is clean and efficient. If you want to make it less verbose, readability (and performance) will suffer.
You may use a helper function which checks if a pointer value is nil:
func n(i interface{}) bool {
v := reflect.ValueOf(i)
return v.Kind() == reflect.Ptr && v.IsNil()
}
And using it:
func check(f *Foo) error {
switch {
case n(f.Bar):
return errors.New("Missing 'bar'")
case n(f.Qux):
return errors.New("Missing 'qux'")
case n(f.Baz):
x := int64(42)
f.Baz = &x
}
return nil
}
Or using if statements:
func check2(f *Foo) error {
if n(f.Bar) {
return errors.New("Missing 'bar'")
}
if n(f.Qux) {
return errors.New("Missing 'qux'")
}
if n(f.Baz) {
x := int64(42)
f.Baz = &x
}
return nil
}
Try these on the Go Playground.
I'm new to Go and I'm learning how to work with goroutines.
I have a function that downloads images:
func imageDownloader(uri string, filename string) {
fmt.Println("starting download for ", uri)
outFile, err := os.Create(filename)
defer outFile.Close()
if err != nil {
os.Exit(1)
}
client := &http.Client{}
req, err := http.NewRequest("GET", uri, nil)
resp, err := client.Do(req)
defer resp.Body.Close()
if err != nil {
panic(err)
}
header := resp.ContentLength
bar := pb.New(int(header))
rd := bar.NewProxyReader(resp.Body)
// and copy from reader
io.Copy(outFile, rd)
}
When I call by itself as part of another function, it downloads images completely and there is no truncated data.
However, when I try to modify it to make it a goroutine, images are often truncated or zero length files.
func imageDownloader(uri string, filename string, wg *sync.WaitGroup) {
...
io.Copy(outFile, rd)
wg.Done()
}
func main() {
var wg sync.WaitGroup
wg.Add(1)
go imageDownloader(url, file, &wg)
wg.Wait()
}
Am I using WaitGroups incorrectly? What could cause this and how can I fix it?
Update:
Solved it. I had placed the wg.add() function outside of a loop. :(
While I'm not sure exactly what's causing your issue, here's two options for how to get it back into working order.
First, looking to the example of how to use waitgroups from the sync library, try calling defer wg.Done() at the beginning of your function to ensure that even if the goroutine ends unexpectedly, that the waitgroup is properly decremented.
Second, io.Copy returns an error that you're not checking. That's not great practice anyway, but in your particular case it's preventing you from seeing if there is indeed an error in the copying routine. Check it and deal with it appropriately. It also returns the number of bytes written, which might help you as well.
Your example doesn't have anything obviously wrong with its use of WaitGroups. As long as you are calling wg.Add() with the same number as the number of goroutines you launch, or incrementing it by 1 every time you start a new goroutine, that should be correct.
However you call os.Exit and panic for certain errors conditions in the goroutine, so if you have more than one of these running, a failure in any one of them will terminate all of them, regardless of the use of WaitGroups. If it's failing without a panic message, I would take a look at the os.Exit(1) line.
It would also, be good practice in go to use defer wg.Done() at the start of your function, so that even if an error occurs, the goroutine still decrements its counter. That way your main thread won't hang on completion if one of the goroutines returns an error.
One change I would make in your example is leverage defer when you are Done. I think this defer ws.Done() should be the first statement in your function.
I like WaitGroup's simplicity. However, I do not like that we need to pass the reference to the goroutine because that would mean that the concurrency logic would be mixed with your business logic.
So I came up with this generic function to solve this problem for me:
// Parallelize parallelizes the function calls
func Parallelize(functions ...func()) {
var waitGroup sync.WaitGroup
waitGroup.Add(len(functions))
defer waitGroup.Wait()
for _, function := range functions {
go func(copy func()) {
defer waitGroup.Done()
copy()
}(function)
}
}
So your example could be solved this way:
func imageDownloader(uri string, filename string) {
...
io.Copy(outFile, rd)
}
func main() {
functions := []func(){}
list := make([]Object, 5)
for _, object := range list {
function := func(obj Object){
imageDownloader(object.uri, object.filename)
}(object)
functions = append(functions, function)
}
Parallelize(functions...)
fmt.Println("Done")
}
If you would like to use it, you can find it here https://github.com/shomali11/util