Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(uat): implement T17 and T18 scenario outlines #348

Merged
merged 19 commits into from
Jul 21, 2023
Merged

feat(uat): implement T17 and T18 scenario outlines #348

merged 19 commits into from
Jul 21, 2023

Conversation

bgklika
Copy link

@bgklika bgklika commented Jun 28, 2023 •
edited
Loading

Issue #, if available:
Implement GGMQ scenario based on GGAD-1-T17, GGAD-1-T18

Description of changes:

  • Add GGMQ-1-T17
  • Add GGMQ-1-T18
  • Tune existing recipe files
  • Update README.md
  • Update CodeBuild spec and github workflow to pass cucumbers tags
  • Update CodeBuild spec by adding console log to artifacts, and cucumber results to reports
  • Tune scenario Examples formatting

Why is this change necessary:
Make replacement of GGAD-1-T17 and GGAD-1-T18

How was this change tested:
Run scenario on CodeBuild

Test results:
T18 (Now clients reduced to only sdk-java)

[INFO ] 2023-06-28 19:12:54.568 [main] StepTrackingReporting - Passed: 'GGMQ-1-T18-v3-sdk-java: As a customer, I can configure IoT Core messages to be forwarded to local MQTT topic, with Greengrass in disconnected state'
[INFO ] 2023-06-28 19:12:54.568 [main] StepTrackingReporting - Passed: 'GGMQ-1-T18-v3-mosquitto-c: As a customer, I can configure IoT Core messages to be forwarded to local MQTT topic, with Greengrass in disconnected state'
[INFO ] 2023-06-28 19:12:54.569 [main] StepTrackingReporting - Passed: 'GGMQ-1-T18-v3-paho-java: As a customer, I can configure IoT Core messages to be forwarded to local MQTT topic, with Greengrass in disconnected state'
[INFO ] 2023-06-28 19:12:54.569 [main] StepTrackingReporting - Passed: 'GGMQ-1-T18-v5-sdk-java: As a customer, I can configure IoT Core messages to be forwarded to local MQTT topic, with Greengrass in disconnected state'
[ERROR] 2023-06-28 19:12:54.569 [main] StepTrackingReporting - Failed: 'GGMQ-1-T18-v5-mosquitto-c: As a customer, I can configure IoT Core messages to be forwarded to local MQTT topic, with Greengrass in disconnected state': Failed at 'the Greengrass deployment is COMPLETED on the device after 5 minutes'
[INFO ] 2023-06-28 19:12:54.569 [main] StepTrackingReporting - Passed: 'GGMQ-1-T18-v5-paho-java: As a customer, I can configure IoT Core messages to be forwarded to local MQTT topic, with Greengrass in disconnected state'

T17:

[INFO ] 2023-06-30 03:58:43.667 [main] StepTrackingReporting - Passed: 'GGMQ-1-T17-v3-sdk-java: As a customer, I can configure IoT Core messages to be forwarded to local MQTT topic, with Greengrass in offline st
ate'
[INFO ] 2023-06-30 03:58:43.668 [main] StepTrackingReporting - Passed: 'GGMQ-1-T17-v5-sdk-java: As a customer, I can configure IoT Core messages to be forwarded to local MQTT topic, with Greengrass in offline st
ate'

Any additional information or context required to review the change:
These scenario outlines failed on Windows due to bug in OTF/Nucleus described here and tagged with @SkipOnWindows

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@bgklika bgklika requested a review from MikeDombo June 28, 2023 17:22
@bgklika bgklika force-pushed the GGMQ-199-implement-ggmq-scenario-based-on-ggad-1-t-17-ggad-1-t-18 branch from 66d5fc4 to bd9ff12 Compare June 28, 2023 17:30
@github-actions
Copy link

github-actions bot commented Jun 28, 2023 •
edited
Loading

Unit Tests Coverage Report

File Coverage Lines Branches
All files 72% 79% 65% ✅
com.aws.greengrass.clientdevices.auth.PermissionEvaluationUtils 78% 82% 74% ✅
com.aws.greengrass.clientdevices.auth.PermissionEvaluationUtils$Operation 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.PermissionEvaluationUtils$Resource 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.CertificateManager 79% 90% 69% ✅
com.aws.greengrass.clientdevices.auth.ClientDevicesAuthService 78% 90% 67% ✅
com.aws.greengrass.clientdevices.auth.DeviceAuthClient 73% 83% 64% ✅
com.aws.greengrass.clientdevices.auth.certificate.ClientCertificateGenerator 95% 90% 100% ✅
com.aws.greengrass.clientdevices.auth.certificate.CertificateHelper 74% 93% 54% ✅
com.aws.greengrass.clientdevices.auth.certificate.CertificateHelper$ProviderType 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.certificate.CertificateStore 72% 85% 60% ✅
com.aws.greengrass.clientdevices.auth.certificate.CertificateExpiryMonitor 77% 87% 67% ✅
com.aws.greengrass.clientdevices.auth.certificate.ServerCertificateGenerator 93% 87% 100% ✅
com.aws.greengrass.clientdevices.auth.certificate.CertificateGenerator 70% 90% 50% ✅
com.aws.greengrass.clientdevices.auth.certificate.CertificateStore$CAType 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.certificate.CertificateExpiryMonitor$CertRotationDecider 90% 100% 80% ✅
com.aws.greengrass.clientdevices.auth.certificate.CertificatesConfig 100% 100% 100% ✅
com.aws.greengrass.clientdevices.auth.session.events.SessionCreationEvent$SessionCreationStatus 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.iot.usecases.VerifyIotCertificate 94% 88% 100% ✅
com.aws.greengrass.clientdevices.auth.iot.usecases.VerifyThingAttachedToCertificate 92% 96% 88% ✅
com.aws.greengrass.clientdevices.auth.iot.usecases.CreateIoTThingSession 87% 90% 83% ✅
com.aws.greengrass.clientdevices.auth.iot.usecases.VerifyCertificateValidityPeriod 88% 88% 0% ✅
com.aws.greengrass.clientdevices.auth.certificate.infra.ClientCertificateStore 100% 100% 100% ✅
com.aws.greengrass.clientdevices.auth.certificate.infra.BackgroundCertificateRefresh 83% 85% 82% ✅
com.aws.greengrass.clientdevices.auth.iot.infra.ThingRegistry 92% 97% 88% ✅
com.aws.greengrass.clientdevices.auth.certificate.usecases.ConfigureManagedCertificateAuthority 85% 85% 0% ✅
com.aws.greengrass.clientdevices.auth.certificate.usecases.ConfigureCustomCertificateAuthority 83% 83% 0% ✅
com.aws.greengrass.clientdevices.auth.certificate.usecases.RegisterCertificateAuthorityUseCase 65% 81% 50% ✅
com.aws.greengrass.clientdevices.auth.configuration.MetricsConfiguration 83% 100% 67% ✅
com.aws.greengrass.clientdevices.auth.configuration.AuthorizationPolicyStatement$Effect 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.configuration.GroupManager 89% 94% 83% ✅
com.aws.greengrass.clientdevices.auth.configuration.ConfigurationFormatVersion 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.configuration.CAConfiguration 96% 100% 92% ✅
com.aws.greengrass.clientdevices.auth.configuration.RuntimeConfiguration 84% 99% 70% ✅
com.aws.greengrass.clientdevices.auth.configuration.SecurityConfiguration 80% 93% 67% ✅
com.aws.greengrass.clientdevices.auth.configuration.CDAConfiguration 100% 100% 100% ✅
com.aws.greengrass.clientdevices.auth.configuration.GroupDefinition 75% 100% 50% ✅
com.aws.greengrass.clientdevices.auth.configuration.ExpressionVisitor 84% 94% 75% ✅
com.aws.greengrass.clientdevices.auth.configuration.GroupConfiguration 90% 95% 86% ✅
com.aws.greengrass.clientdevices.auth.api.ServiceErrorEvent 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.api.ClientDevicesAuthServiceApi 90% 79% 100% ✅
com.aws.greengrass.clientdevices.auth.api.DomainEvents 100% 100% 100% ✅
com.aws.greengrass.clientdevices.auth.api.AuthorizeClientDeviceActionEvent$AuthorizationStatus 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.api.UseCases 71% 92% 50% ✅
com.aws.greengrass.clientdevices.auth.api.DomainEvent 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.api.GetCertificateRequestOptions$CertificateType 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.session.attribute.StringLiteralAttribute 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.session.attribute.WildcardSuffixAttribute 88% 100% 75% ✅
com.aws.greengrass.clientdevices.auth.certificate.events.CertificateSubscriptionEvent$SubscriptionStatus 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.certificate.events.CACertificateChainChanged 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.iot.Certificate$Status 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.iot.IotAuthClient$Default 56% 47% 64% ✅
com.aws.greengrass.clientdevices.auth.iot.Thing 82% 86% 79% ✅
com.aws.greengrass.clientdevices.auth.iot.Certificate 78% 89% 67% ✅
com.aws.greengrass.clientdevices.auth.iot.GreengrassV2DataClientFactory 18% 18% 0% ❌
com.aws.greengrass.clientdevices.auth.iot.CertificateRegistry 95% 90% 100% ✅
com.aws.greengrass.clientdevices.auth.iot.Component 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.iot.events.VerifyClientDeviceIdentityEvent$VerificationStatus 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.infra.NetworkStateProvider$Default$1 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.infra.NetworkStateProvider$ConnectionState 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.infra.NetworkStateProvider$Default 75% 90% 60% ✅
com.aws.greengrass.ipc.IPCUtils 83% 67% 100% ✅
com.aws.greengrass.ipc.VerifyClientDeviceIdentityOperationHandler 60% 69% 50% ✅
com.aws.greengrass.ipc.GetClientDeviceAuthTokenOperationHandler 86% 98% 75% ✅
com.aws.greengrass.ipc.AuthorizeClientDeviceActionOperationHandler 79% 92% 67% ✅
com.aws.greengrass.ipc.SubscribeToCertificateUpdatesOperationHandler 81% 88% 75% ✅
com.aws.greengrass.clientdevices.auth.session.SessionConfig 92% 100% 83% ✅
com.aws.greengrass.clientdevices.auth.session.SessionManager$1 100% 100% 100% ✅
com.aws.greengrass.clientdevices.auth.session.MqttSessionFactory 100% 100% 100% ✅
com.aws.greengrass.clientdevices.auth.session.SessionCreator 100% 100% 100% ✅
com.aws.greengrass.clientdevices.auth.session.SessionManager 88% 100% 75% ✅
com.aws.greengrass.clientdevices.auth.session.SessionImpl 100% 100% 100% ✅
com.aws.greengrass.clientdevices.auth.session.SessionCreator$SessionFactorySingleton 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.session.MqttSessionFactory$MqttCredential 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.certificate.handlers.CACertificateChainChangedHandler 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.certificate.handlers.CAConfigurationChangedHandler 93% 87% 100% ✅
com.aws.greengrass.clientdevices.auth.certificate.handlers.CertificateRotationHandler 96% 91% 100% ✅
com.aws.greengrass.clientdevices.auth.certificate.handlers.SecurityConfigurationChangedHandler 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.metrics.handlers.SessionCreationEventHandler 88% 100% 75% ✅
com.aws.greengrass.clientdevices.auth.metrics.handlers.MetricsConfigurationChangedHandler 70% 90% 50% ✅
com.aws.greengrass.clientdevices.auth.metrics.handlers.AuthorizeClientDeviceActionsMetricHandler 88% 100% 75% ✅
com.aws.greengrass.clientdevices.auth.metrics.handlers.VerifyClientDeviceIdentityEventHandler 88% 100% 75% ✅
com.aws.greengrass.clientdevices.auth.metrics.handlers.CertificateSubscriptionEventHandler 83% 100% 67% ✅
com.aws.greengrass.clientdevices.auth.metrics.handlers.ServiceErrorEventHandler 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.iot.dto.CertificateV1DTO$Status 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.connectivity.usecases.GetConnectivityInformationUseCase 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.connectivity.usecases.RecordConnectivityChangesUseCase 100% 100% 100% ✅
com.aws.greengrass.clientdevices.auth.util.ResizableLinkedBlockingQueue 90% 80% 100% ✅
com.aws.greengrass.clientdevices.auth.util.ParseIPAddress 90% 95% 84% ✅
com.aws.greengrass.clientdevices.auth.metrics.ClientDeviceAuthMetrics 87% 98% 75% ✅
com.aws.greengrass.clientdevices.auth.metrics.MetricsEmitter 100% 100% 100% ✅
com.aws.greengrass.clientdevices.auth.connectivity.CISShadowMonitor 63% 77% 50% ✅
com.aws.greengrass.clientdevices.auth.connectivity.RecordConnectivityChangesResponse 100% 100% 100% ✅
com.aws.greengrass.clientdevices.auth.connectivity.HostAddress 67% 67% 0% ✅
com.aws.greengrass.clientdevices.auth.connectivity.RecordConnectivityChangesRequest 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.connectivity.ConnectivityInformation 100% 100% 100% ✅
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionConstants 100% 100% 0% ✅
com.aws.greengrass.clientdevices.auth.configuration.parser.TokenMgrError 22% 32% 12% ❌
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionTokenManager 61% 65% 58% ✅
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTStart 33% 33% 0% ❌
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTAnd 67% 67% 0% ✅
com.aws.greengrass.clientdevices.auth.configuration.parser.Token 58% 58% 0% ✅
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionDefaultVisitor 0% 0% 0% ❌
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTOr 67% 67% 0% ✅
com.aws.greengrass.clientdevices.auth.configuration.parser.SimpleCharStream 28% 31% 25% ❌
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpressionTreeConstants 0% 0% 0% ❌
com.aws.greengrass.clientdevices.auth.configuration.parser.JJTRuleExpressionState 67% 65% 70% ✅
com.aws.greengrass.clientdevices.auth.configuration.parser.ASTThing 67% 67% 0% ✅
com.aws.greengrass.clientdevices.auth.configuration.parser.RuleExpression 63% 63% 62% ✅
com.aws.greengrass.clientdevices.auth.configuration.parser.SimpleNode 27% 35% 19% ❌

Minimum allowed coverage is 50%

Generated by 🐒 cobertura-action against 75a4014

@bgklika bgklika marked this pull request as draft June 29, 2023 13:40
@bgklika bgklika marked this pull request as ready for review June 30, 2023 02:10
@bgklika bgklika changed the title feat(uat): implement GGMQ-1-T18 scenario outline feat(uat): implement T17 and T18 scenario outlines Jun 30, 2023
@bgklika bgklika requested a review from jcosentino11 July 3, 2023 11:47
@bgklika bgklika force-pushed the GGMQ-199-implement-ggmq-scenario-based-on-ggad-1-t-17-ggad-1-t-18 branch from 4c0d552 to a4ed4d5 Compare July 3, 2023 14:17
@bgklika bgklika marked this pull request as draft July 3, 2023 15:33
@bgklika
Copy link
Author

bgklika commented Jul 4, 2023

PR moved to Draft due to pending for changes from aws-greengrass/aws-greengrass-testing#211 should be delivered first.

@bgklika bgklika force-pushed the GGMQ-199-implement-ggmq-scenario-based-on-ggad-1-t-17-ggad-1-t-18 branch from a4ed4d5 to 60fcb07 Compare July 5, 2023 16:44
@bgklika bgklika marked this pull request as ready for review July 5, 2023 16:51
@bgklika bgklika marked this pull request as draft July 5, 2023 19:45
@bgklika bgklika force-pushed the GGMQ-199-implement-ggmq-scenario-based-on-ggad-1-t-17-ggad-1-t-18 branch from 60fcb07 to 4a91e0c Compare July 7, 2023 15:18
@bgklika bgklika marked this pull request as ready for review July 7, 2023 15:19
@bgklika bgklika requested a review from MikeDombo July 10, 2023 16:28
@bgklika bgklika force-pushed the GGMQ-199-implement-ggmq-scenario-based-on-ggad-1-t-17-ggad-1-t-18 branch from 4a91e0c to b858dc8 Compare July 10, 2023 16:33
@bgklika bgklika marked this pull request as draft July 10, 2023 18:13
@bgklika bgklika force-pushed the GGMQ-199-implement-ggmq-scenario-based-on-ggad-1-t-17-ggad-1-t-18 branch 2 times, most recently from cf8844c to d9fb3c7 Compare July 19, 2023 18:20
@bgklika bgklika marked this pull request as ready for review July 19, 2023 18:20
@bgklika bgklika requested a review from MikeDombo July 19, 2023 18:21
MikeDombo
MikeDombo previously approved these changes Jul 19, 2023
View reviewed changes
@MikeDombo
Copy link
Member

OTF tests seem maybe stuck? It has been running nearly 3 hours now.

@bgklika
Copy link
Author

bgklika commented Jul 19, 2023

Locally it extend 3 hours. On CodeBuild should run faster but now we have a lot of scenarios to run.

@bgklika
Copy link
Author

bgklika commented Jul 19, 2023

3h 41m 3s

@bgklika
feat(uat): add GGMQ-1-T18 scenario outline
38ff651
@bgklika
feat(uat): tune T18 and remove unused code
465b95c
@bgklika
fix(uat): remove mosquitto and paho clients from T18
7736487
@bgklika
fix(uat): remove comment lines
2da13b3
@bgklika
feat(uat): add T17 scenario outline with restart greengrass
b5b0439
@bgklika
fix(uat): remove extra checks from platform code; tune network utils …
35bcdf7 
…steps
@bgklika
fix(uat): resolve bug when SDK client is not terminated on failed con…
28f7edb 
…nect()
@bgklika
feat(uat): add LogSteps and use it
8b2bb0f
@bgklika
fix(uat): remove sudo from linux iptables command
5a8497c
@bgklika
fix(uat): remove code due to moved to OTF
44aae13
@bgklika
fix(uat): move NetworkUtilsSteps to OTF
c328e53
@bgklika
fix(uat): tune scenarios
43a61b4
@bgklika
fix(uat): remove Bridge component jar and recipe and use local config…
649ee4f 
…uraion update step
@bgklika
fix(uat): update uat/README.md
560f3bc
@bgklika
feat(uat): extend T17 and T18 for all clients
3518f1d
@bgklika
fix(uat): move T17 and T18 before T20
54d5311
@bgklika
feat(uat): pass cucumber tags from github workflow; run T17/18 for on…
d077f40 
…ly SDK client
@bgklika
fix(uat): reorder GGMQ-1-Tx scenarios
75a4014
@bgklika bgklika force-pushed the GGMQ-199-implement-ggmq-scenario-based-on-ggad-1-t-17-ggad-1-t-18 branch from d9fb3c7 to 75a4014 Compare July 21, 2023 17:19
@bgklika bgklika requested a review from MikeDombo July 21, 2023 17:24
@MikeDombo MikeDombo merged commit bb149d0 into uat-dev Jul 21, 2023
7 checks passed
@MikeDombo MikeDombo deleted the GGMQ-199-implement-ggmq-scenario-based-on-ggad-1-t-17-ggad-1-t-18 branch July 21, 2023 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MikeDombo MikeDombo MikeDombo approved these changes

@jcosentino11 jcosentino11 Awaiting requested review from jcosentino11

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bgklika @MikeDombo