CI / CD Process Update: Preparation and Planning

Classic introduction

1. CruiseControl.NET was used as a build system. Working with him is a dubious pleasure in principle. XML configs on multiple screens, a bunch of dependencies and linking configs to each other, lack of modern solutions to modern problems.
2. Some developers (mainly leads) on the project must have access to build servers, and they sometimes like to change ccnet configs which should not be changed. Guess what happens next. You need to have a simple and convenient management of rights in the CI system, without taking away from the developers access to the server. Simply put, there is no text config - there is nowhere to climb with playful hands.
3. - … CCNET, git. (. ).
4. , . , . 
5. - , — ( ) .
6. . ? ? ? .
7. Suboptimal use of resources and an outdated process of building and delivering code.

1. Easy maintenance with all the consequences (backups, updates, problem solving, if any).
2. Universality. Ideally, develop a general scheme according to which all modules are assembled and delivered, and use it as a template.
3. Minimize time for adding new build configurations and maintenance. Clients are added / removed. Sometimes it becomes necessary to make new setups. It is necessary that when creating a new module there should be no delay with the configuration of its delivery.

Preparation: repository conversion

1. We take the extension mercurial hggit as a basis. 
2. We get a list of all branches of the mercurial repository. 
3. We convert branch names (thanks to mercurial and the developers for the spaces in the branch names, special thanks for the umlauts and other characters that add joy to life).
4. Make bookmarks (hg bookmark) and push into the intermediate repository. What for? Because bitbucket, and because you cannot create a bookmark with the same name as the name of the branch (for example, staging). 
5. In the new (already git) repository, remove the postfix from the branch name and migrate hgignore to gitignore. 
6. Push to the main repository.

1. $ cd /path/to/hg/repo
   $ cat << EOF >> ./.hg/hgrc
   [extensions]
   hggit=
   EOF
   

2. $ hg branches > ../branches
   

3. #!/usr/bin/env bash
   hgBranchList="./branches"
   sed -i 's/:.*//g' ${hgBranchList}
   symbolsToDelete=$(awk '{print $NF}' FS=" " ${hgBranchList} > sym_to_del.tmp)
    
   i=0
   while read hgBranch; do
     hgBranches[$i]=${hgBranch}
     i=$((i+1))
   done <${hgBranchList}
    
   i=0
   while read str; do
     strToDel[$i]=${str}
     i=$((i+1))
   done < ./sym_to_del.tmp
    
   for i in ${!strToDel[@]}
   do
     echo ${hgBranches[$i]} | sed "s/${strToDel[$i]}//" >> result.tmp
   done
    
   sed -i 's/[ \t]*$//' result.tmp
   sed 's/^/"/g' result.tmp > branches_hg
   sed -i 's/$/"/g' branches_hg
   sed 's/ /-/g' result.tmp > branches_git
   sed -i 's/-\/-/\//g' branches_git
   sed -i 's/-\//\//g' branches_git
   sed -i 's/\/-/\//g' branches_git
   sed -i 's/---/-/g' branches_git
   sed -i 's/--/-/g' branches_git
   rm sym_to_del.tmp
   rm result.tmp
   

4. #!/usr/bin/env bash
   gitBranchList="./branches_git"
   hgBranchList="./branches_hg"
   hgRepo="/repos/reponame"
    
   i=0
   while read hgBranch; do
     hgBranches[$i]=${hgBranch}
     i=$((i+1))
   done <${hgBranchList}
    
   i=0
   while read gitBranch; do
     gitBranches[$i]=${gitBranch}
     i=$((i+1))
   done <${gitBranchList}
    
   cd ${hgRepo}
   for i in ${!gitBranches[@]}
   do
     hg bookmark -r "${hgBranches[$i]}" "${gitBranches[$i]}-cnv"
   done
    
   hg push git+ssh://git@bitbucket.org:username/reponame-temp.git
   echo "Done."
   

5. #!/bin/bash
   # clone repo, run git branch -a, delete remotes/origin words to leave only branch names, delete -cnv postfix, delete default branch string because we can't delete it
   repo="/repos/repo"
   gitBranchList="./branches_git"
   defaultBranch="default-cnv"
   while read gitBranch; do   gitBranches[$i]=${gitBranch};   i=$((i+1)); done < $gitBranchList
   cd $repo
   for i in ${!gitBranches[@]}; do git checkout ${gitBranches[$i]}-cnv; done
   git checkout $defaultBranch
   for i in ${!gitBranches[@]}; do
       git branch -m ${gitBranches[$i]}-cnv ${gitBranches[$i]}
       git push origin :${gitBranches[$i]}-cnv ${gitBranches[$i]}
       git push origin -u ${gitBranches[$i]}
   done
   

6. #!/bin/bash
   # clone repo, run git branch -a, delete remotes/origin words to leave only branch names, delete -cnv postfix, delete default branch string because we can't delete it
   repo="/repos/repo"
   gitBranchList="./branches_git"
   defaultBranch="default"
   while read gitBranch; do   gitBranches[$i]=${gitBranch};   i=$((i+1)); done < $gitBranchList
   cd $repo
   for i in ${!gitBranches[@]}; do
       git checkout ${gitBranches[$i]}
       sed -i '1d' .hgignore
       mv .hgignore .gitignore
       git add .
       git commit -m "Migrated ignore file"
   done
   

Conclusion

• preparation (demo to the customer using an example of one build, software installation, repository conversion, updating existing ccnet configs);
• CI / CD configuration for dev environment and its tests (the old system remains working in parallel);
• CI / CD settings for the remaining environments (in parallel with the existing "good") and tests;
• migration dev, staging and test;
• Migrating production and transferring domains from old IIS instances to new ones.