Hook up Pod Status check with Skaffold.

[tejal29]

Fixes: #3915
Relates to: #3739, #3813

This PR, hook up validator.Pod with skaffold. I wanted some early feedback on the output and code.

Note:

1. This is Large PR with number of changes/ refactors
2. Could use some more testing.

Sample outputs

1. skaffold run on microservices example when a deploy is succssesful

[microservices:]$~/workspace/skaffold/out/skaffold run -d gcr.io/tejal-test --status-check
Generating tags...
 ...
Checking cache...
 - leeroy-web: Found Locally
 - leeroy-app: Found Locally
Tags used in deployment:
...
Starting deploy...
  - deployment/leeroy-app: waiting for rollout to finish: 0 of 1 updated replicas are available...
    - pod/leeroy-app-5547fb9bbd-8q8sz: pod is in phase Pending due to reason ContainerCreating.
    - pod/leeroy-app-56676c994c-952rp: pod is stable
 - deployment/leeroy-web: waiting for rollout to finish: 0 of 1 updated replicas are available...
    - pod/leeroy-web-6d9b5cf6f7-d7pgk: pod is in phase Pending due to reason ContainerCreating.
    - pod/leeroy-web-7b7f5dfb8-2ntzn: pod is stable
 - deployment/leeroy-web is ready.
 - deployment/leeroy-app is ready. [1/2 deployment(s) still pending]
Deployments stabilized in 2.960726657s
You can also run [skaffold run --tail] to get the logs

2. skaffold run when there is an error in the config.

Starting deploy...
WARN[0000] image [leeroy-app] is not used by the deployment 
 - deployment.apps/leeroy-web created
 - service/leeroy-app created
 - deployment.apps/leeroy-app created
Waiting for deployments to stabilize...
 - deployment/leeroy-app: waiting for rollout to finish: 0 of 1 updated replicas are available...
    - pod/leeroy-app-696dc76bfc-gsmm7: pod is in phase Pending due to reason ContainerCreating.
 - deployment/leeroy-web: waiting for rollout to finish: 0 of 1 updated replicas are available...
    - pod/leeroy-web-6c977597dc-v8gc7: pod is in phase Pending due to reason ContainerCreating.
 - deployment/leeroy-web is ready. [1/2 deployment(s) still pending]
 - deployment/leeroy-app: waiting for rollout to finish: 0 of 1 updated replicas are available...
    - pod/leeroy-app-696dc76bfc-gsmm7: pod is in phase Pending due to reason ErrImagePull due to rpc error: code = Unknown desc = Error response from daemon: pull access denied for leeroy-app1, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
 - deployment/leeroy-app: waiting for rollout to finish: 0 of 1 updated replicas are available...
    - pod/leeroy-app-696dc76bfc-gsmm7: pod is in phase Pending due to reason ImagePullBackOff due to Back-off pulling image "leeroy-app1".
...
 - deployment/leeroy-app: waiting for rollout to finish: 0 of 1 updated replicas are available...
    - pod/leeroy-app-696dc76bfc-gsmm7: pod is in phase Pending due to reason ErrImagePull due to rpc error: code = Unknown desc = Error response from daemon: pull access denied for leeroy-app1, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
 - deployment/leeroy-app failed. Error: could not stabilize within 2m0s: context deadline exceeded.

To try out

1. Checkout this Pr
2. Run make
3. Use the following command

skaffold run -d  $PROJECT --status-check

[codecov]

Codecov Report

Merging #3917 into master will decrease coverage by 0.99%.
The diff coverage is 4.31%.

Impacted Files	Coverage Δ
pkg/diag/diag.go	0.00% <0.00%> (ø)
pkg/diag/validator/pod.go	0.00% <0.00%> (ø)
pkg/diag/validator/resource.go	0.00% <0.00%> (ø)
pkg/skaffold/deploy/resource/pod.go	37.03% <0.00%> (-62.97%)	⬇️
pkg/skaffold/deploy/status_check.go	57.55% <16.21%> (-10.76%)	⬇️
...affold/kubernetes/portforward/kubectl_forwarder.go	65.85% <0.00%> (-2.44%)	⬇️