Skip to content

test: implement the cucumber conformance tests - #238

Merged
lance merged 1 commit into
cloudevents:masterfrom
lance:71-add-cucumber-conformance-tests
Jul 13, 2020
Merged

lance merged 1 commit into
cloudevents:masterfrom
lance:71-add-cucumber-conformance-tests

Conversation

@lance

@lance lance commented Jul 2, 2020 •

Copy link
Copy Markdown
Member

Fixes: #71

Items remaining

  • use git submodule to pull in the features from cloudevents/spec
  • investigate why time stamps are not identical
  • investigate why datacontenttype is not identical
  • validate event data

@lance lance self-assigned this Jul 3, 2020
@lance lance added module/test Issues and changes related to the tests spec/1.0 Issues related to the CE specification type/enhancement New feature or request labels Jul 3, 2020
@lance
lance marked this pull request as ready for review July 3, 2020 15:23
@lance
lance requested review from a team, danbev, helio-frota and lholmquist and removed request for a team July 3, 2020 15:23

@helio-frota helio-frota left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conformance tests are passing which is great!
But I'm not following why conformance submodule directory needs to have a golang project inside:

13:50 $ tree conformance/
conformance/
├── cmd
│   └── cloudevents
│       └── main.go
├── features
│   ├── http-protocol-binding.feature
│   └── kafka-protocol-binding.feature
├── go.mod
├── go.sum
├── hack
│   └── presubmit-test.sh
├── LICENSE
├── pkg
│   ├── commands
│   │   ├── commands.go

I'm guess that is because the (updated / common used) conformance spec is there.
If yes, maybe it can be downloaded and / or use the online version ?

It works fine (package.json):

"conformance": "cucumber-js https://raw.githubusercontent.com/cloudevents/conformance/master/features/http-protocol-binding.feature -p default",
13:51 $ npm run conformance

> cloudevents-sdk@2.0.2 conformance /home/hf/code/cloudevents/sdk-javascript
> cucumber-js https://raw.githubusercontent.com/cloudevents/conformance/master/features/http-protocol-binding.feature -p default

@http
Feature: HTTP protocol binding

  @http
  Scenario Outline: Binary content mode (application/json)
    Given HTTP Protocol Binding is supported
    Given an HTTP request
      """
      POST /someresource HTTP/1.1
...

@lance

lance commented Jul 3, 2020

Copy link
Copy Markdown
Member Author

@helio-frota that's a good suggestion. I didn't realize you could provide a URL. Definitely better than pulling in a submodule

@lance

lance commented Jul 6, 2020

Copy link
Copy Markdown
Member Author

@helio-frota I have not been able to make your suggestion work. Unclear why. 🤷‍♂️

@helio-frota

Copy link
Copy Markdown
Member

@lance that was a false positive : ( the submodule was here, and when running pointing to a remote file, the cucumber falls back to the local file in the disk. Sorry.

But to try to remedy that I have created a small package called downtotemp and you can try using this way:

"conformance": "npx downtotemp https://raw.githubusercontent.com/cloudevents/conformance/master/features/http-protocol-binding.feature && cucumber-js /tmp/http-protocol-binding.feature -p default",

It is working least.

This commit adds cucumber-js conformance steps and includes the cucumber
tests in 'npm test'.

Signed-off-by: Lance Ball <lball@redhat.com>
@lance
lance force-pushed the 71-add-cucumber-conformance-tests branch from 3307be8 to f639fb6 Compare July 6, 2020 21:49
@lance

lance commented Jul 7, 2020

Copy link
Copy Markdown
Member Author

@cloudevents/sdk-javascript-maintainers PTAL

@helio-frota thanks for the suggestions - done.

@lance lance changed the title test: initial pass at implementing the cucumber conformance tests test: implement the cucumber conformance tests Jul 7, 2020
@lance

lance commented Jul 8, 2020

Copy link
Copy Markdown
Member Author

Ping @cloudevents/sdk-javascript-maintainers ptal

@helio-frota
helio-frota self-requested a review July 8, 2020 18:52

@lholmquist lholmquist left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reasoning for moving the tests to an integration directory? Will the move mess with the merging of #233 since those tests are being updated there?

@lance

lance commented Jul 8, 2020

Copy link
Copy Markdown
Member Author

@lholmquist it's mainly to keep the integration tests separate from the conformance steps that are now in test/conformance. I'm not sure how much trouble that merge will be. I imagine it won't be a problem, but 🤷

@lholmquist lholmquist left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welp, LGTM then

@lance
lance merged commit dca2811 into cloudevents:master Jul 13, 2020
@lance
lance deleted the 71-add-cucumber-conformance-tests branch July 13, 2020 13:47
@lance lance mentioned this pull request Jul 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module/test Issues and changes related to the tests spec/1.0 Issues related to the CE specification type/enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Integrate with conformance cucumber tests

4 participants