Skip to content
This repository was archived by the owner on Aug 29, 2025. It is now read-only.

First draft of New Connection Documentation - #331

Merged
n-riesco merged 48 commits into
plotly:masterfrom
shannonlal:master
Jan 17, 2018
Merged

n-riesco merged 48 commits into
plotly:masterfrom
shannonlal:master

Conversation

@shannonlal

Copy link
Copy Markdown
Contributor

This is a first draft of the new Connection document as discussed in issue #266. I have tried to break down all the major steps that are required to create a new connection. I would welcome some comments and insight on how to make this document better. In addition, I have put some instructions on how to run the development environment as TBD. If someone can add some updated instructions that would be great.

Comment thread NEW_CONNECTION.md Outdated
The following are the instructions for adding a new data source

## Add the following files for Creating a new Datasource

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd format this as a list and perhaps have two sections: one for new files and another for updated files.

Comment thread NEW_CONNECTION.md Outdated


## Images
It is recommended to add the images as png and a good idea to include a regular and small icon

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add locations were logos are currently stored.

Comment thread NEW_CONNECTION.md Outdated
function query(queryObject, connection);

//Question are these required?
function files(connection);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove (normal connectors don't use this)

Comment thread NEW_CONNECTION.md
function files(connection);

function storage(connection);
```

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove (normal connectors don't use this)

@n-riesco

Copy link
Copy Markdown
Contributor

I'll give you more feedback after Jan, 3rd

@shannonlal

Copy link
Copy Markdown
Contributor Author

I cleaned up some instructions to broke it down into lists as suggested. A couple of points that I think would help myself and other developers.

  1. Instructions on how to setup the development and debugging environment.
  2. I am still working my way through the Athena connector and at this point I have not had to add anything to the Settings.react.js file. I am not sure if I have missed anything at this point but any advice would be greatly appreciated.

@shannonlal

Copy link
Copy Markdown
Contributor Author

Just a small update for the Settings.react.js file. I figured out there was an update that needed to be added to this file to make the query work correctly

@chriddyp

chriddyp commented Jan 3, 2018

Copy link
Copy Markdown
Member

Could we update CONTRIBUTING.md with a link to this file?

Comment thread NEW_CONNECTION.md Outdated
# Detailed Implementation Instructions

## Images
It is recommended to add the images as png and a good idea to include a regular and small icon.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't use the small icons any longer.
We only need an icon that looks good when displayed with max-width: 100px; and max-height: 70px;.

Comment thread NEW_CONNECTION.md Outdated
1. Add an entry in DIALECTS
2. Add an entry in SQL_DIALECTS_USING_EDITOR
3. Add an entry under CONNECTION_CONFIG
4. Add the location of the logo in

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Add the location of the logo into LOGOS

Comment thread NEW_CONNECTION.md Outdated
3. Add an entry under CONNECTION_CONFIG
4. Add the location of the logo in
5. Add the preview query in the PREVIEW_QUERY section
6. Add an entry in the SAMPLE_DBS

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Add sample credentials for your connector in the SAMPLE_DBS

Comment thread NEW_CONNECTION.md
## Updating the Tab.react.js
The following are the instructions for adding the Tab React JS file
1. Add an entry for the connection label

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Customise the label for your connector here:

        let label;
        if (dialect === DIALECTS.S3) {
            label = `S3 - (${connectionObject.bucket})`;
        } else if (dialect === DIALECTS.APACHE_DRILL) {
            label = `Apache Drill (${connectionObject.host})`;
        } else if (dialect === DIALECTS.APACHE_IMPALA) {
              label = `Apache Impala (${connectionObject.host}:${connectionObject.port})`;
        } else if (dialect === DIALECTS.APACHE_SPARK) {
            label = `Apache Spark (${connectionObject.host}:${connectionObject.port})`;
        } else if (connectionObject.dialect === DIALECTS.ELASTICSEARCH) {
            label = `Elasticsearch (${connectionObject.host})`;
        } else if (connectionObject.dialect === DIALECTS.SQLITE) {
            label = connectionObject.storage;
        } else {
            label = `${connectionObject.database} (${connectionObject.username}@${connectionObject.host})`;
        }

Comment thread NEW_CONNECTION.md
Currently there is Dialect types for SQL, ElasticSearch, S3 and Apache Drill. If
your new data source is an SQL Type you will need to add the new Dialect in the
fetchData section for SQL types.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should clean up this file.

For the time being, I'd keep things simple and just ask the developer to update fetchData and add their connector dialect to the list in:

        if (contains(connectionObject.dialect, [
                    DIALECTS.APACHE_IMPALA,
                    DIALECTS.APACHE_SPARK,
                    DIALECTS.IBM_DB2,
                    DIALECTS.MYSQL, DIALECTS.MARIADB, DIALECTS.POSTGRES,
                    DIALECTS.REDSHIFT, DIALECTS.MSSQL, DIALECTS.SQLITE,
                    DIALECTS.DATA_WORLD
        ])) {

Comment thread NEW_CONNECTION.md Outdated
* -row[1] column name
* -row[2] data type
* @param {Object} connection
* @return {Array}

@n-riesco n-riesco Jan 12, 2018 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

 * @returns {Promise} that resolves to { columnnames: [ 'tablename', 'column_name', 'data_type' ], rows: [[tablename1, columnname1, datatype1], ...]] }

Comment thread NEW_CONNECTION.md

/**
* The following function will connect to the datasource
* @param {Object} connection

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

 * @returns {Promise} that resolves when the connection succeeds

Comment thread NEW_CONNECTION.md Outdated
* The following function will return a list of tables.
* The return result should an array of strings
* @param connection {object}
* @return {Array}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

 * @returns {Promise} that resolves to an array of table names

Comment thread NEW_CONNECTION.md Outdated
* The second is a two dimensional array which represents the rows to be displayed
* @param {string|object} queryObject
* @param {object} connection
* @return {object} -- tuple {columnames, rows}

@n-riesco n-riesco Jan 12, 2018 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

 * @returns {Promise} that resolves to { columnnames, rows }

@n-riesco

Copy link
Copy Markdown
Contributor

@shannonlal I've just noticed that you've started to implement the Athena connector.

Shall we change the title of this PR to reflect the change?

@shannonlal

Copy link
Copy Markdown
Contributor Author

@n-riesco. Thanks for the comments I can look at these tonight. With regards to the renaming the pull request. You can leave this as is. For the Athena stuff. I have the backend working and just now have to integrate it into the frontend. I should have this ready to go in about a week

@shannonlal

Copy link
Copy Markdown
Contributor Author

@n-riesco. I noticed that the build is failing due to an issue with DB2 Unit Tests. I think you had mentioned that this was not related to my work. Is this till the case? Is there anything else you need from me for this pull request? The Athena stuff I will do on a separate pull request.

@n-riesco

Copy link
Copy Markdown
Contributor

@shannonlal For a documentation change, it's OK to ignore CircleCI. But if you want to get rid of the failure, the only thing you need to do is to rebase this PR against master.

@n-riesco

Copy link
Copy Markdown
Contributor

I'll have a look at this PR later today.

Comment thread CONTRIBUTING.md Outdated
Builds are uploaded to https://github.com/plotly/falcon-sql-client/releases.

## Troubleshooting
The Falcon Configuration information is installed in the users home directory.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the user's home directory.

Comment thread CONTRIBUTING.md Outdated

## Troubleshooting
The Falcon Configuration information is installed in the users home directory.
For example Unix and Mac (~/.plotly/connector). If you have tried the install

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Windows %userprofile%\.plotly\connector\

Comment thread NEW_CONNECTION.md Outdated
# Add New Data Source

## Images
It is recommended to add the images as png and a good idea to include.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Icons

The connector must provide an icon to be displayed on the selection tab.

Comment thread NEW_CONNECTION.md Outdated
```

## Updating the constants.js file
The following are the instructions for adding the Constants File:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/adding/updating/

Comment thread NEW_CONNECTION.md Outdated
6. Add sample credentials for your connector in the SAMPLE_DBS

## Updating the Tab.react.js
The following are the instructions for adding the Tab React JS file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/adding/updating/

Comment thread NEW_CONNECTION.md Outdated

## Updating the Settings.react.js
Update the fetchData function to make the calls to appropriate sessions.js backend.
Add your new connectino to the following list:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/connectino/connection/

@n-riesco

Copy link
Copy Markdown
Contributor

@shannonlal I've made a few small comments after a quick read. I'll merge the PR after you address those.

Again, thank you for writing this up!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants