how-to-safely-append-data-to-the-same-slice-concurrently-in-golang

发表于
活着是世界上最罕见的事，

大多数人只是存在，仅此而已。

In this article, we’ll go through different ways to implement like wait groups and mutex and the challenges of data race.

Aim : A program that separates odd and even number from 0to 9 an appends it into their corresponding slices. So we should have odd=[1,3,5,7,9] (in any order) and even=[0,2,4,6,8] (in any order).

Attempt-0 Only with goroutines

Background : The only goroutine that a program has at startup is the one that calls the main function, thus we refer to it as the main goroutine. The go statement generates new goroutines. A go statement is a regular function or method call that has the keyword go prefixed to it. A go statement causes a newly formed goroutine to call the function. The go statement itself is immediately finished.

In this we are appending to the slice using multiple go routines. So we’ll have a main go routines and 10 new go routines created by anonymous go function.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
package main
import (
 "fmt"
)
func done() {
var odd = make([]int, 0)
 var even = make([]int, 0)
for i := 0; i <= 9; i++ {
  if i%2 == 0 {
   go func(i int) {
    even = append(even, i)
   }(i)
  } else {
   go func(i int) {
    odd = append(odd, i)
   }(i)
  }
 }
 fmt.Println(odd)
 fmt.Println(even)
}
func main() {
 for i := 1; i <= 10; i++ {
  fmt.Println("========================")
  done()
 }
}

Output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
========================
[1 3 5 7]               
[0 2 8]                 
========================
[7 5 3]                 
[4 6 0 2 8]             
========================
[1 3 5]                 
[0 2 4 6 8]             
========================
[1 3 5 7]               
[0 2 4 6 8]             
========================
[1 3 5]                 
[0 2 4 6 8]             
========================
[1 3 5 7]               
[0 2 4 6 8]             
========================
[1 3 5 7]               
[0 2 4 6 8]             
========================
[1 3 5]                 
[0 2 4 6 8]             
========================
[1 3 5 7]
[0 2 4 6 8]
========================
[1 3 5 7]
[0 2 4 6 8]

Major issues :

  • The main goroutine can complete it’s execution without waiting for the other go routines (who will append the data in slices)to complete. As a result print statement in the main go routine will the slices where the data is still being appended by other go routines.
  • Data race (we’ll cover in the next part)

Attempt-1 With sync.waitgroups

We’ll use sync.waitgroups. With the help of the WaitGroup function in the sync package, a program can wait for particular goroutines. These Golang sync techniques halt programme execution until goroutines in the WaitGroup have finished running.

In a nutshell, the main program will wait till all the other go routines have finished.

WaitGroup is informed by wg.Add(1) that it needs to wait for one more goroutine every time loop starts. After that, defer wg.Done() alerts the WaitGroup when a goroutine is finished. Then, wg.Wait() delays the execution until the goroutines have finished running. The entire procedure resembles increasing a counter in wgAdd() deducts from the wg counter. Waiting for the counter in wg to reach 0 while calling done(). This is the crux how wait groups work.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
package main
import (
 "fmt"
 "sync"
)
func done() {
var wg sync.WaitGroup
 var odd = make([]int, 0)
 var even = make([]int, 0)
for i := 0; i <= 9; i++ {
  wg.Add(1)
  if i%2 == 0 {
   go func(i int) {
    defer wg.Done()
    even = append(even, i)
   }(i)
  } else {
   go func(i int) {
    defer wg.Done()
    odd = append(odd, i)
   }(i)
  }
 }
 wg.Wait()
 fmt.Println(odd)
 fmt.Println(even)
}
func main() {
 for i := 1; i <= 10; i++ {
  fmt.Println("========================")
  done()
 }
}

Output

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
========================
[1 3 5 7 9]             
[0 2 4 6 8]             
========================
[1 3 9]                 
[0 2 4 6 8]             
========================
[9 5 7 1 3]             
[6 8 0 2 4]             
========================
[1 3 5 7 9]             
[0 2 4 6 8]             
========================
[1 3 5 7 9]             
[0 2 4 8]               
========================
[1 3 5 9 7]             
[0 2 4 6 8]             
========================
[1 3 5 7 9]             
[0 2 4 6 8]             
========================
[9 1 3 5 7]             
[0 2 4 6 8]             
========================
[3 9 1 5 7]
[0 4 2 6 8]
========================
[9 1 7 3 5]
[0 6 8 2 4]

What’s happening now?
To make sure the function was consistent, we ran it ten times. It undoubtedly outperforms the old one, but occasionally the results are not what was anticipated. So what’s the reason. The reason is data race .

Data race : When two or more Goroutines access the same memory location, at least one of them is a write, and there is no ordering between them, this is referred to as a data race.

In simpler terms, you have a race condition with multiple goroutines writing a slice concurrently. The behavior will be unpredictable. You need a mutex. Protect the appends by lock.

We can confirm this by command go run -race pgm3b.go

Attempt-2 With sync.waitgroups and mutex

We’ll use mutex with sync.waitgroups. This not only ensures that the main go routine waits till all other 10 go routines complete but it also ensures that at a given point of time only one go routine can write in a slice. This is the principle of mutual exclusion or mutex. Appends are protected by mutex avoiding dirty write case.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
package main
import (
 "fmt"
 "sync"
)
func done() {
type answer struct {
  MU   sync.Mutex
  data []int
 }
var odd answer
 var even answer
wg := &sync.WaitGroup{}
 for i := 0; i <= 9; i++ {
  if i%2 == 0 {
   wg.Add(1)
   go func(i int) {
    defer wg.Done()
even.MU.Lock()
    even.data = append(even.data, i)
    even.MU.Unlock()
   }(i)
  } else if i%2 == 1 {
   wg.Add(1)
   go func(i int) {
    defer wg.Done()
odd.MU.Lock()
    odd.data = append(odd.data, i)
    odd.MU.Unlock()
   }(i)
  }
 }
wg.Wait()
fmt.Println(odd.data)
fmt.Println(even.data)
}
func main() {
 for i := 1; i <= 10; i++ {
  fmt.Println("========================")
  done()
}
}

output

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
========================
[1 9 5 3 7]             
[0 4 2 6 8]             
========================
[5 9 3 7 1]             
[2 0 8 6 4]             
========================
[1 3 5 7 9]             
[2 0 4 6 8]             
========================
[1 3 5 7 9]             
[0 2 4 6 8]             
========================
[1 3 5 7 9]             
[0 2 4 6 8]             
========================
[1 3 5 7 9]             
[2 0 4 6 8]             
========================
[1 3 5 7 9]             
[0 2 4 6 8]             
========================
[3 1 9 7 5]             
[0 2 4 6 8]             
========================
[9 5 7 1 3]
[6 8 4 2 0]
========================
[1 3 5 7 9]
[0 2 4 6 8]

https://blog.devgenius.io/how-to-safely-append-data-to-the-same-slice-concurrently-in-golang-df467e1ebc9c