Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/fabric builder k8s #739

Merged
merged 3 commits into from
May 26, 2022
Merged

Feature/fabric builder k8s #739

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f2a923a
Support the fabric-builder-k8s for the chaincode "easy button."
jkneubuh May 13, 2022
a7a4705
Run the CI/CD test suite using the correct matrix env for k8s builder
jkneubuh May 19, 2022
d8540ee
Incorporate review feedback : remove zz_unused and pkgk8scc routines
jkneubuh May 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions ci/azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ jobs:
Basic-Typescript:
CHAINCODE_NAME: basic
CHAINCODE_LANGUAGE: typescript

steps:
- template: templates/install-deps.yml
- script: sudo apt-get install softhsm2
Expand All @@ -135,12 +136,17 @@ jobs:
vmImage: ubuntu-20.04
strategy:
matrix:
Typescript-Java:
CCaaS-Java:
CLIENT_LANGUAGE: typescript
CHAINCODE_LANGUAGE: java
Typescript-Golang:
CCaaS-Golang:
CLIENT_LANGUAGE: typescript
CHAINCODE_LANGUAGE: external
K8s-Builder-Java:
CHAINCODE_NAME: basic
CHAINCODE_LANGUAGE: java
CHAINCODE_BUILDER: k8s

steps:
- template: templates/install-k8s-deps.yml
- script: ../ci/scripts/run-k8s-test-network-basic.sh
Expand Down
3 changes: 2 additions & 1 deletion ci/scripts/run-k8s-test-network-basic.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ set -euo pipefail
export CONTAINER_CLI=${CONTAINER_CLI:-docker}
export CLIENT_LANGUAGE=${CLIENT_LANGUAGE:-typescript}
export CHAINCODE_LANGUAGE=${CHAINCODE_LANGUAGE:-java}
export TEST_NETWORK_CHAINCODE_BUILDER=${CHAINCODE_BUILDER:-ccaas}

# Fabric version and Docker registry source: use the latest stable tag image from JFrog
export FABRIC_VERSION=${FABRIC_VERSION:-2.4}
Expand Down Expand Up @@ -70,7 +71,7 @@ function createNetwork() {
./network channel create

print "Deploying chaincode"
./network chaincode deploy asset-transfer-basic basic_1.0 $TEST_NETWORK_CHAINCODE_PATH
./network chaincode deploy $CHAINCODE_NAME $TEST_NETWORK_CHAINCODE_PATH
}

function stopNetwork() {
Expand Down
1 change: 1 addition & 0 deletions test-network-k8s/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ network-debug.log
build/
.env
bin/
*.tgz
2 changes: 1 addition & 1 deletion test-network-k8s/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Launch the network, create a channel, and deploy the [basic-asset-transfer](../a

./network channel create

./network chaincode deploy asset-transfer-basic basic_1.0 $PWD/../asset-transfer-basic/chaincode-java
./network chaincode deploy asset-transfer-basic ../asset-transfer-basic/chaincode-java
```

Invoke and query chaincode:
Expand Down
17 changes: 12 additions & 5 deletions test-network-k8s/config/org1/core.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -558,11 +558,18 @@ chaincode:
# chaincode. The external builder detection processing will iterate over the
# builders in the order specified below.
externalBuilders:
- name: ccaas_builder
path: /opt/hyperledger/ccaas_builder
propagateEnvironment:
- CHAINCODE_AS_A_SERVICE_BUILDER_CONFIG

- name: ccaas_builder
path: /opt/hyperledger/ccaas_builder
propagateEnvironment:
- CHAINCODE_AS_A_SERVICE_BUILDER_CONFIG

# copied into the /var persistent volume share by "network up"
- name: k8s_builder
path: /var/hyperledger/fabric/external_builders/k8s_builder
propagateEnvironment:
- CORE_PEER_ID
- KUBERNETES_SERVICE_HOST
- KUBERNETES_SERVICE_PORT

# The maximum duration to wait for the chaincode build and install process
# to complete.
Expand Down
18 changes: 13 additions & 5 deletions test-network-k8s/config/org2/core.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -558,11 +558,19 @@ chaincode:
# chaincode. The external builder detection processing will iterate over the
# builders in the order specified below.
externalBuilders:
- name: ccaas_builder
path: /opt/hyperledger/ccaas_builder
propagateEnvironment:
- CHAINCODE_AS_A_SERVICE_BUILDER_CONFIG

- name: ccaas_builder
path: /opt/hyperledger/ccaas_builder
propagateEnvironment:
- CHAINCODE_AS_A_SERVICE_BUILDER_CONFIG

# copied into the /var persistent volume share by "network up"
- name: k8s_builder
path: /var/hyperledger/fabric/external_builders/k8s_builder
propagateEnvironment:
- CORE_PEER_ID
- KUBERNETES_SERVICE_HOST
- KUBERNETES_SERVICE_PORT

# The maximum duration to wait for the chaincode build and install process
# to complete.
installTimeout: 300s
Expand Down
4 changes: 2 additions & 2 deletions test-network-k8s/docs/CHAINCODE_AS_A_SERVICE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ network up
network channel create
```
```
network chaincode deploy asset-transfer-basic basic_1.0 ${PWD}
network chaincode deploy asset-transfer-basic ${PWD}
```
```
network chaincode metadata asset-transfer-basic
Expand All @@ -55,7 +55,7 @@ kind load docker-image fabric-samples/asset-transfer-basic/chaincode-java

```shell
# Assemble the chaincode package archive
network chaincode package basic_1.0 asset-transfer-basic $PWD/build/asset-transfer.tgz
network chaincode package asset-transfer-basic asset-transfer-basic $PWD/build/asset-transfer.tgz

# Determine the ID for the chaincode package
CORE_CHAINCODE_ID_NAME=$(network chaincode id $PWD/build/asset-transfer.tgz)
Expand Down
23 changes: 23 additions & 0 deletions test-network-k8s/kube/fabric-builder-role.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: fabric-builder-role
rules:
- apiGroups:
- ""
- apps
resources:
- pods
- deployments
- configmaps
- secrets
verbs:
- get
- watch
- create
18 changes: 18 additions & 0 deletions test-network-k8s/kube/fabric-builder-rolebinding.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: fabric-builder-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: fabric-builder-role
subjects:
- namespace: ${NS}
kind: ServiceAccount
name: default
33 changes: 33 additions & 0 deletions test-network-k8s/kube/org1/org1-install-k8s-builder.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
---
apiVersion: batch/v1
kind: Job
metadata:
name: org1-install-k8s-builder
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sure there's a good reason but I'm slightly puzzled by the approach of using a peer image which already has the k8s builder preconfigured, just to copy out the builder binaries for use by the peer image which the k8s peer image is based on

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is a "good reason" but I ran into a case yesterday where I needed to test some changes in the fabric-builder-k8s binaries. I was queued up with some code to switch over to the wget/curl style download from the binary release archive, and then realized that this meant there was no way for me to test my builder updates without publishing custom binaries to a web server. AND the only way to build a Linux binary is to ... run linux ... or wade through Golang args to figure out how to cross-compile the builders natively on my machine. AND ... no makefile in the builder package ... AND ... I.e.. the only way to test an edit to the builder is to publish a release on the builder project and let the GitHub action turn the crank.

Seems like a little thing but it spun out of control pretty quickly. I just wanted to test out a one line update in the golang code, but couldn't do so without a few hours of hacking on the environment.

So the workaround that seemed to do well was to use the Docker build of the builders, tag it, and load it into KIND so that the cluster would not go out to pull an image (or binary) from ghcr.io. This only worked with the distribution of binaries via Docker. Something like:

cd fabric-builder-k8s 

docker build -t ghcr.io/hyperledgendary/k8s-fabric-peer:v0.4.0-joshfancy .

kind load docker-image ghcr.io/hyperledgendary/k8s-fabric-peer:v0.4.0-joshfancy

and ...

cd test-network-k8s 

export TEST_NETWORK_K8S_CHAINCODE_BUILDER_VERSION=v0.4.0-joshfancy

...

So some work in this area -- somewhere -- still needs to be done. I agree it is a bit unconventional with the current scheme but it seemed to work out well for the task at hand.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was more that k8s-fabric-peer already has everything you need in it as-is without any messing about with wget, which is why I added/use TEST_NETWORK_FABRIC_PEER_IMAGE all the time. I'm sure I'm just missing something obvious.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aha!

The line in the sand here is : "no Kube dependencies in Fabric." (No fair building out of-band peer images that pre-bundle the builder.) The builder is either IN the peer image, or it's NOT in the peer image.

Answer: it's NOT. This PR works around the constraint of "no builder in peer."

Let's not re-visit / open : fabric #3405 or fabric #3407

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a difference between putting the k8s builder code in the fabric repo with the CCaaS builder, which seems like a fair line in the sand to preserve, and publishing a handy preconfigured peer image from the k8s builder repo. I don't mind either way but there is no "the peer image" but rather a sample peer image provided "out of the box" by Fabric, which as far as I know is not actually intended for serious use. Other peer images are available, such as the very nice k8s builder peer image :)

spec:
backoffLimit: 0
completions: 1
template:
metadata:
name: org1-install-k8s-builder
spec:
restartPolicy: "Never"
containers:
- name: main
image: ghcr.io/hyperledgendary/k8s-fabric-peer:${K8S_CHAINCODE_BUILDER_VERSION}
imagePullPolicy: IfNotPresent
command:
- sh
- -c
- "mkdir -p /mnt/fabric-org1/fabric/external_builders && cp -rv /opt/hyperledger/k8s_builder /mnt/fabric-org1/fabric/external_builders/"
volumeMounts:
- name: fabric-org1-volume
mountPath: /mnt/fabric-org1
volumes:
- name: fabric-org1-volume
persistentVolumeClaim:
claimName: fabric-org1
33 changes: 33 additions & 0 deletions test-network-k8s/kube/org2/org2-install-k8s-builder.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
---
apiVersion: batch/v1
kind: Job
metadata:
name: org2-install-k8s-builder
spec:
backoffLimit: 0
completions: 1
template:
metadata:
name: org2-install-k8s-builder
spec:
restartPolicy: "Never"
containers:
- name: main
image: ghcr.io/hyperledgendary/k8s-fabric-peer:${K8S_CHAINCODE_BUILDER_VERSION}
imagePullPolicy: IfNotPresent
command:
- sh
- -c
- "mkdir -p /mnt/fabric-org2/fabric/external_builders && cp -rv /opt/hyperledger/k8s_builder /mnt/fabric-org2/fabric/external_builders/"
volumeMounts:
- name: fabric-org2-volume
mountPath: /mnt/fabric-org2
volumes:
- name: fabric-org2-volume
persistentVolumeClaim:
claimName: fabric-org2
4 changes: 3 additions & 1 deletion test-network-k8s/network
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,10 @@ export CLUSTER_NAME=${TEST_NETWORK_KIND_CLUSTER_NAME:-kind}
export NS=${TEST_NETWORK_KUBE_NAMESPACE:-${NETWORK_NAME}}
export DOMAIN=${TEST_NETWORK_DOMAIN:-vcap.me}
export CHANNEL_NAME=${TEST_NETWORK_CHANNEL_NAME:-mychannel}
export ORDERER_TIMEOUT=${TEST_NETWORK_ORDERER_TIMEOUT:-10s} # see https://github.com/hyperledger/fabric/issues/3372
export ORDERER_TIMEOUT=${TEST_NETWORK_ORDERER_TIMEOUT:-10s} # see https://github.com/hyperledger/fabric/issues/3372
export TEMP_DIR=${PWD}/build
export CHAINCODE_BUILDER=${TEST_NETWORK_CHAINCODE_BUILDER:-ccaas} # see https://github.com/hyperledgendary/fabric-builder-k8s/blob/main/docs/TEST_NETWORK_K8S.md
export K8S_CHAINCODE_BUILDER_VERSION=${TEST_NETWORK_K8S_CHAINCODE_BUILDER_VERSION:-v0.4.0}

LOG_FILE=${TEST_NETWORK_LOG_FILE:-network.log}
DEBUG_FILE=${TEST_NETWORK_DEBUG_FILE:-network-debug.log}
Expand Down
Loading