Clash Royale CLAN TAG#URR8PPP

How to update value of two struct efficiently

I have the following code which parses YAML files and needs to match values from one struct external and update the internal struct's type property.

external

internal

type

For example, this is the yaml file(translated to bin for simplicity) and content which is parsed correctly

package main

import (
"fmt"
"gopkg.in/yaml.v2"
"log"
)

//internal config model for parsing

type InternalModel struct
models Model2 `yaml:"models"`


type Model2 struct
Name string `yaml:"name"`
Type string `yaml:"type"`
Target string `yaml:"target"`


var internal_config = byte(`
models:
- name: myapp
type: app1
target: ./

- name: myapp2
type: app2
target: ./
`)

type ExternalConfig struct
Landscape Zone `yaml:"Landscape"`


type Zone struct
Zone string `yaml:"zone"`
Models Model `yaml:"models"`


type Model struct
AppType string `yaml:"app-type"`
ServiceType string `yaml:"service-type"`



var external_config = byte(`
Landscape:
zone: zone1
models:
- app-type: app1
service-type: GCP
- app-type: app2
service-type: AMAZON
zone: zone2
models:
- app-type: app3
service-type: AZURE
- app-type: app4Í
service-type: HEROKU
`)

func main()

// This is the internal config which needs updated

internalConfiglYaml := InternalModel

err := yaml.Unmarshal(byte(internal_config), &internalConfiglYaml)
if err != nil
log.Fatalf("error in model internalConfiglYaml: %v", err)

//fmt.Printf("%+vn", internalConfiglYaml)

//--------------------------Second config file-----------------------//
//This is the external config yaml

extConfigYaml := ExternalConfig

err = yaml.Unmarshal(byte(external_config), &extConfigYaml)
if err != nil
log.Fatalf("error in model extConfigYaml: %v", err)

fmt.Printf("%+vn", extConfigYaml)

landscape := "zone1"

modifiedConfig := ConvertTypes(internalConfiglYaml, extConfigYaml, landscape)

fmt.Printf("%+vn", modifiedConfig)



func ConvertTypes(int_cfg InternalModel, ext_config ExternalConfig, landscape string) (out_cfg InternalModel)

for _, module := range int_cfg.models

switch module.Type
case "app1":
//here I hard-coded the value "GCP" but it should come from the yaml struct after parsing
module.Type = "GCP" // should be something like ext_config.models.service-type when the key in the struct
case "app2":
//here I hard-coded the value "AMAZON" but it should come from the yaml struct after parsing
module.Type = "AMAZON"




return int_cfg


//At the end what I need to do is to get the internal yaml file to be changed to the following struct
//The changes are when the type=app-type I need to modify the type in the internal config, here its GCP and ruby

//internal_config_after_changes := byte(`
//
//
//models:
// - name: myapp
// type: GCP
// target: ./
//
// - name: myapp2
// type: AMAZON
// target: ./
//
//
//`)


At the end what I need to do is to get the internal yaml file to be changed to the struct above internal_config_after_changes
The changes are when the type=app-type I need to modify the type value in the internal_config, here from app1 to GCP and app2 to amazon

internal_config_after_changes

type=app-type

type

internal_config

app1

GCP

app2

amazon

The problem is with the second loop which I should use to iterate on the external_config and the matching values, I'm not sure how to combine them both with efficient way...

external_config

internal_config_after_changes

external_config

app1

ConvertType

4 Answers
4

Golang FAQ described regarding pointers to maps and slice:

Map and slice values behave like pointers: they are descriptors that
contain pointers to the underlying map or slice data. Copying a map or
slice value doesn't copy the data it points to. Copying an interface
value makes a copy of the thing stored in the interface value. If the
interface value holds a struct, copying the interface value makes a
copy of the struct. If the interface value holds a pointer, copying
the interface value makes a copy of the pointer, but again not the
data it points to.

On iterating through the slice of model inside ConvertType you are actually creating a copy of Models slice whose value.Type is not changing the value of original struct due that reason.

ConvertType

Models

for _, module := range int_cfg.models


Above code snippet is creating a copy of int_cfg.models.

int_cfg.models

Index the slice model to point to the exact underlying array of slice Model to change the value as:

package main

import (
"fmt"
"log"
"strings"

"gopkg.in/yaml.v2"
)

//internal config model for parsing

type InternalModel struct
Models Model2 `yaml:"models"`


type Model2 struct
Name string `yaml:"name"`
Type string `yaml:"type"`
Target string `yaml:"target"`


var internal_config = byte(`
models:
- name: myapp
type: app1
target: ./

- name: myapp2
type: app2
target: ./
`)

type ExternalConfig struct
Landscape Zone `yaml:"Landscape"`


type Zone struct
Zone string `yaml:"zone"`
Models Model `yaml:"models"`


type Model struct
AppType string `yaml:"app-type"`
ServiceType string `yaml:"service-type"`


var external_config = byte(`
Landscape:
- zone: zone1
models:
- app-type: app1
service-type: GCP
- app-type: app2
service-type: AMAZON
- zone: zone2
models:
- app-type: app3
service-type: AZURE
- app-type: app4Í
service-type: HEROKU
`)

func main()

//This is the internal config which needs to be update

internalConfiglYaml := InternalModel

err := yaml.Unmarshal(internal_config, &internalConfiglYaml)
if err != nil
log.Fatalf("error in model internalConfiglYaml: %v", err)

fmt.Printf("%+vn", internalConfiglYaml)

//--------------------------Second config file-----------------------//
//This is the external config yaml

extConfigYaml := ExternalConfig
// var response interface

err = yaml.Unmarshal(external_config, &extConfigYaml)
if err != nil
log.Fatalf("error in model extConfigYaml: %v", err)

fmt.Printf("%+vn", extConfigYaml)

landscape := "zone1"

modifiedConfig := ConvertTypes(&internalConfiglYaml, extConfigYaml, landscape)

fmt.Printf("%+vn", modifiedConfig)



// ConvertTypes for changing the intConfig struct types
func ConvertTypes(int_cfg *InternalModel, ext_config ExternalConfig, landscape string) (out_cfg *InternalModel)

for _, module := range ext_config.Landscape
if module.Zone == landscape
for i, value := range module.Models
switch strings.Compare(value.AppType, int_cfg.Models[i].Type)
case 0:
//here I hard-coded the value "GCP" but it should come from the yaml struct after parsing
int_cfg.Models[i].Type = value.ServiceType // should be something like ext_config.models.service-type when the key in the struct
default:





return int_cfg



If you check above code snippet, you will also recognize that I have changed the struct.

type InternalModel struct
models Model2 `yaml:"models"`



to first letter uppercase to make it exportable as:

type InternalModel struct
Models Model2 `yaml:"models"`



Because of the struct InternalModel is unexportable field model was unable to parse the provided internal_config yaml which leads to empty slice data after unmarshal yaml.

InternalModel

model

internal_config

One more thing that I have noticed is you are converting bytes into bytes again. There is no need for that.

err := yaml.Unmarshal(byte(internal_config), &internalConfiglYaml)


So I have changed it to just:

err := yaml.Unmarshal(internal_config, &internalConfiglYaml)


Since internal_config is already declared as byte using byte in global variable.

internal_config

byte

GCP

amazon

extrnal_config

If you're looking for how to apply the app-type to service-type mapping from the ExternalConfig, I suggest converting the array into a map and using it to look up the mapping:

ExternalConfig

func ConvertTypes(intCfg *InternalModel, extCfg ExternalConfig, landscape string)
// Make an app-type to service-type map.
appToSvc := make(map[string]string, len(extCfg.Landscape.Models))
for _, m := range extCfg.Landscape.Models
appToSvc[m.AppType] = m.ServiceType


// For each InternalModel model, check if there's an app-type to service-type
// mapping. If so, use the service-type instead of the app-type.
for i, model := range intCfg.Models
if t, ok := appToSvc[model.Name]; ok
intCfg.Models[i].Type = t





I've also fixed the pass-by-pointer and range loop assignment problems as suggested by @Himanshu and @Zan Lynx.

fixed the pass-by-pointer and range loop assignment problems

The first thing to consider is that you are working with two structures that are defined at run-time, thus you will not be able to use a single loop to match them. The next thing to consider is that your InternalConfig is the one least likely to change and should thus be the first structure to parse, where after the ExternalConfig is parsed and matched with it.

From reading your question, comments and the provided answers it also seems that there are a number of assumptions on the board which few have alluded to. I will attempt to address these if possible.

So let's give it a go:

package main

import (
// import standard library packages first (alphabetically)
"fmt"
"log"

// import third party packages next (also alphabetically)
"gopkg.in/yaml.v2"

// finally import your own packages here
)

// InternalConfig
// - Let's call it InternalConfig so as to ensure consistent naming standards
type InternalConfig struct
Models Imodel `yaml:"models"`
// Exported to ensure access to external packages (such as yaml)


// Imodel - The internal model struct
type Imodel struct
Name string `yaml:"name"`
Type string `yaml:"type"`
Target string `yaml:"target"`


var iCfgb = byte(`
models:
- name: myapp
type: app1
target: ./

- name: myapp2
type: app2
target: ./
`)

type ExternalConfig struct
Landscape Zone `yaml:"Landscape"`


type Zone struct
Zone string `yaml:"zone"`
Models Emodel `yaml:"models"`


type Emodel struct
AppType string `yaml:"app-type"`
ServiceType string `yaml:"service-type"`



var eCfgb = byte(`
Landscape:
zone: zone1
models:
- app-type: app1
service-type: GCP
- app-type: app2
service-type: AMAZON
zone: zone2
models:
- app-type: app3
service-type: AZURE
- app-type: app4Í
service-type: HEROKU
`)

function main()
iCfgYaml := InternalConfig
// don't need byte(iCfgb) as iCfgb is already a byte slice
// We are also using a shorthand if statement as we will only use err in the
// if scope, this is because our config is passed by reference and not value
if err := yaml.Unmarshal(iCfgb, &iCfgYaml); err != nil
log.Fatalf("error in model InternalConfig: %v", err)


eCfgYaml := ExternalConfig
if err := yaml.Unmarshal(eCfgb, &eCfgYaml); err != nil
log.Fatalf("error in model ExternalConfig: %v", err)


// From here there are clearly some assumptions made, here is my take:
// 1. I do not see any purpose in the landscape variable, the value "zone1"
// can be obtained from eCfgYaml.Landscape.Zone;
// 2. From the question it seems you want to update the InternalConfig
// values with values from the ExternalConfig. However, from the code you
// created a modified copy. I will assume this was for debugging purposes
// and skip the creating of another copy;

// I have renamed ConvertTypes to UpdateInternalState to accurately reflect
// the behavior of the function. It now returns an error instead of a copy of
// InternalConfig (any state altering functions should most likely return an
// error). It receives a pointer of InternalConfig and a value copy of
// ExternalConfig. This is because we will be altering the state of
// InternalConfig
if err := UpdateInternalState(&iCfgYaml, eCfgConfig); err != nil
log.Fatalf("Error updating internal config: %v", err)


// Assume the print is for debugging, thus I denote it with DBG. Makes it easy
// to find and remove from production code.
fmt.Printf("DBG: %+vn", iCfgYaml)


func UpdateInternalState(iCfg *InternalConfig, eCfg ExternalConfig) error
// Firstly, we need to track the internal config models. We will create an
// index lookup map (ilm) for it.
ilm := make(map[string]int)

// Now we iterate through our internal models and update the index:
for i, v := range iCfg.Models
ilm[v] = i


// Now we iterate through the external config and update the internal config
// state accordingly:
for _, em := range eCfg.Landscape.Models
// We find the index of the specified app type in the index lookup
// We use a shorthand if statement as we will only use i and ok within
// the scope of the if (if you need it outside you can't use shorthand)
// If there is no match nothing happens, but if found it will update to
// the corresponding specified value in the external config.
if i, ok := ilm[em.AppType]; ok
iCfg.Models[i].Type = em.ServiceType



// At any point within this function you are able to do more robust error
// checking, such as ensuring the index lookup map must have a corresponding
// entry, and return any errors encountered.
return nil



This question is definitely a duplicate but I can't find one to link right now.

Your problem is that the for loop is creating a copy of each struct. Writing into element which is just a copy of xtr[i] never changes the original xtr[i]. What you do to fix it is to use the slice index instead.

element

xtr[i]

xtr[i]

Change

for _, element := range xtr
switch element.vType
case "app1":
//user1 value is coming from the &y parsed data of the yaml
element.vType = "user1"
case "app2":
//user2 value is coming from the &y parsed data
element.vType = "user2"





Into

for i := range xtr
switch xtr[i].vType
case "app1":
//user1 value is coming from the &y parsed data of the yaml
xtr[i].vType = "user1"
case "app2":
//user2 value is coming from the &y parsed data
xtr[i].vType = "user2"





parsed

xtr

user1

user2

&y

app1

both

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.