A small all in one data science tool

dataplay3

Quick Start

To build your own dataplay3 container, run

make docker

and then start the dataplay3 in a container with

make run

open http://localhost:8000 to access dataplay3 ui.

Feature Overview

dataplay3 provides following features:

• Dataset management
• Dataset query
• Data visualization
• A Simple dashboard
• Categorical data prediction
• Numerical data prediction
• Time serials data prediction

refer to feature introduction for details

Architecture

Here is a list of what is being used to build dataplay3:

Client Side

• React A JavaScript library for building user interfaces
• Ant Design Pro Out-of-box UI solution for enterprise applications
  • Ant Design An enterprise-class UI design language and React implementation.
  • UmiJS Pluggable enterprise-level react application framework.
  • Dva Lightweight front-end framework based on redux, redux-saga and react-router
• BizCharts Powerful data visualization library based on G2 and React
  • G2 The Grammar of Graphics in JavaScript

Server Side

• sanic async python 3.5+ web server
• pandas python data analysis library
• auto-sklearn an automated machine learning toolkit
• prophet tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.
• pandassql allows you to query pandas DataFrames using SQL syntax
• gunicorn a python WSGI HTTP Server for UNIX

Here is a component view of dataplay3

@startuml
swag
title Dataplay3 Architecture

package "Client" {
    [Dashboard Page] as dashboard
    [Dataset Page] as dataset
    [Visualization Page] as viz
    [Prediction Page] as ml
    [Configuration Page] as config
    [Services Wrapper] as services
}


package "Server" {
    [REST API] as api
    [Dashboard]
    [Configuration]
    [ML]
    [Dataset]
    [AutoSklearn]
    [Prophet]
    [Pandas]
}

dashboard -down-> services
dataset -down-> services
viz -down-> services
ml -down-> services
config -down-> services

services -down-( api

api -down-> Dashboard
api -down-> Configuration
api -down-> ML
api -down-> Dataset

ML -down-> AutoSklearn
ML -down-> Prophet

Dataset -down-> Pandas

@enduml

Development

Client

Dataplay3 client is based on ant design pro. To build the client, you need have node installed.

To develop the client, first install all dependencies.

cd client
npm install

yarn can be used for dependency management as well.

Start the client

npm run start

To build the client and copy all the output client code to the static directory of the server

npm run build

More client build options refer to dataplay3/client/package.json

Server

Dataplay3 server is based on Python3 and Sanic.

To setup server development environment, you need Python3 and pip. It is recommended to using virtual env to manage your python environment. Run following command to create your python virtual environment.

python -m venv .venv
source .venv/bin/activate

Install development dependency

cd server
pip3 install -r dev-requirements.txt

And then use doit to install all the dependencies for dataplay3 server.

doit install_dep

Depending on your develop operating system, you may need manually install gcc and swig, refer to these how to install gcc, how to install swig

To start the dataplay server, run

doit server

To access open api doc, visit http://localhost:8000/swagger/

for more build options of server, run

doit list