Eskimo is a state of the art Big Data Infrastructure and Management Web Console to build, manage and operate Big Data 2.0 Analytics clusters on Kubernetes. This is the git repository of Eskimo Community Edition.
////
This file is part of the eskimo project referenced at www.eskimo.sh. The licensing information below apply just as
well to this individual file than to the Eskimo Project as a whole.
Copyright 2019 www.eskimo.sh - All rights reserved.
Author : http://www.eskimo.sh
Eskimo is available under a dual licensing model : commercial and GNU AGPL.
If you did not acquire a commercial licence for Eskimo, you can still use it and consider it free software under the
terms of the GNU Affero Public License. You can redistribute it and/or modify it under the terms of the GNU Affero
Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option)
any later version.
Compliance to each and every aspect of the GNU Affero Public License is mandatory for users who did no acquire a
commercial license.
Eskimo is distributed as a free software under GNU AGPL in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Affero Public License for more details.
You should have received a copy of the GNU Affero Public License along with Eskimo. If not,
see https://www.gnu.org/licenses/ or write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
Boston, MA, 02110-1301 USA.
You can be released from the requirements of the license by purchasing a commercial license. Buying such a
commercial license is mandatory as soon as :
The above copyright notice and this licensing notice shall be included in all copies or substantial portions of the
Software.
////
:sectnums:
:authors: www.eskimo.sh
©️ www.eskimo.sh / 2019 - 2023
== Eskimo Introduction
A state of the art Big Data Infrastructure and Management Platform to build, manage and operate
Big Data 2.0 Analytics clusters on Kubernetes.
image::doc/guides/eskimo-guide/pngs/status-wide-condensed.png[800, 800, align=“center”]
Eskimo is in a certain way the Operating System of your Big Data Cluster:
Eskimo is as well:
Eskimo Packages all the components above and takes care of all the configuration and tuning required to make them work
together smoothly and without the user or administrator needing to take care of anything, Big Data the Plug & Play
way.
image::doc/guides/eskimo-guide/pngs/eskimo_platform.png[800, 800, align=“center”]
=== Eskimo CE Project Development Status
GitHub Action CI Integration Build Status +
image:https://github.com/eskimo-sh/eskimo/actions/workflows/maven.yml/badge.svg[“Build Status”, link=“https://github.com/eskimo-sh/eskimo/actions/”]
SonarCloud Quality Checks Status +
image:https://sonarcloud.io/api/project_badges/measure?project=eskimo-sh_eskimo&metric=alert_status[“Sonarcloud Status”, link=“https://sonarcloud.io/dashboard?id=eskimo-sh_eskimo”]
== Information
Reach http://www.eskimo.sh for more information on Eskimo or look at the documentation in the folder doc.
== Building eskimo
=== Requirements
Eskimo uses plain old Apache Maven for building (https://maven.apache.org/) as well as JDK 11+
(https://openjdk.java.net/) for both building and execution.
Minimum requirements are as follows:
Every other dependency will be downloaded during the maven build process (except trilead-ssh2, see <
below).
Theoretically, Eskimo can be built on any Operating System providing a command line and supporting Java 11+
(Note : previous versions of JDK down to 8 may be supported without any guarantee) and Maven 3+.
==== Microsoft Windows
However, the eskimo source tree makes an extensive use of symbolic links which - unfortunately - are only supported on
Microsoft Windows 10 and 11 through “Developer Mode”.
In order to checkout and develop Eskimo on Windows, some precautions need to be taken:
git clone command should explicitly enable symbolic link support. That would mean using followinggit clone -c core.symlinks=true https://github.com/eskimo-sh/eskimo.gitIf these two precautions are well taken, one should be able to develop Eskimo on Windows. +
A significant portion of the unit tests will however be skipped when executed on Windows since they require a Unix
Operating System to work properly.
=== Building using maven
In order to build eskimo, simply run the following command in this very folder (the one containing this readme.adoc
file):
As a result of the build process, Eskimo is available as an extractable archive in eskimo-_version_-bin.zip or
eskimo-_version_-bin.tar.gz in the folder target.
=== Specific hints and troubleshooting information regarding eskimo building
You might want to read carefully the following additional information related to building eskimo.
==== Put maven and Java in PATH
Of course, for the above command to work, you need to have java and mvn in your path.
Use for instance the following commands on Linux:
(You might want to put above commands in your /etc/profile or /etc/bash.bashrc)
And for instance the following commands on Windows
[[trilead-ssh2]]
==== Install maven dependency trilead-ssh2
Eskimo used the library trilead-ssh2 version build-217-jenkins-27 which is not available in standard maven
repositories. +
This library is bundled with the eskimo sources in the folder lib. +
One can install it in his own local maven repository using the following command:
=== Eskimo Build result
The build results at the end of the maven build process is located in the folder `target. +
The results are two archive files:
eskimo-_version_-bin.zipeskimo-_version_-bin.tar.gzYou can find pre-built packages of Eskimo on https://www.eskimo.sh.
The archive contains the full software packages with all dependencies and command line executables required to
start it as well as the full documentation, etc.
== Running Eskimo
This section gives instructions about running eskimo.
Eskimo itself can run on any Operating System supporting a JVM (Linux, Windows, Mac OSX) but cluster nodes are only
supported running Linux.
=== Prerequisites
==== System requirements
In order to run eskimo, one needs to have
(See https://github.com/eskimo-sh/eskimo/blob/master/doc/guides/eskimo-guide/eskimo-guide.adoc[The Eskimo User Guide]
for more information on Eskimo cluster requirements.)
Eskimo is reached using a web browser (see startup logs). Supported web browsers are:
Note: there may be other browsers / versions supported (Safari, Opera but they are not certified to work with Eskimo)
==== Available JVM with Java in PATH
In order to run eskimo, one needs to have java in the path.
Use for instance the following commands on Linux:
(You might want to put above commands in your /etc/profile or /etc/bash.bashrc)
And for instance the following commands on Windows
=== Extract archive.
After building eskimo using maven, the zip and tarball archives are located in the folder target.
One of these archives needs to be extracted on the local filesystem.
Then in the folder bin under the newly extracted eskimo binary distribution folder, one can find two scripts:
eskimo.bat to execute eskimo on Windowseskimo.sh to execute eskimo on Linux.=== Access eskimo
With eskimo properly started using the above scripts, one can reach eskimo using http://machine_ip:9191. +
The default port number is 9191. This can be changed in configuration file eskimo.properties.
The default login / password credentials are admin / password.
=== Typical startup issues
Several issues can happen upon first eskimo startup. +
This section describes common issues and ways to resolved them.
==== eskimo-users.json cannot be written
If you meet an error as the following on startup:
Eskimo uses a local file to define users and access credentials. Upon first startup, if that file doesn’t exist already,
it is created by eskimo (with the default credentials above) at the path pointed to by the property
security.userJsonFile in eskimo.properties.
If you experience the error above or something alike, change that propery to point to a location where the first
version of the file can successfully be created.
=== Building packages / docker images locally
With released versions of Eskimo, operators / administrators have the choice to either build packages / docker images
locally - which boils down to creating container images where the target software (e.g. spark, flink, gluster, etc.) is
downloaded from internet and installed - or download pre-built images from www.eskimo.sh. +
Whereas on development - SNAPSHOT - versions of Eskimo downloading pre-built images is not possible, only the
possibility to build images locally is available.
Whenever building images locally, the majority of the failures from from references versions of software components not
avaiilable anymore. For instance, when the Apache flink team releases a new minor version, they most of the time remove
the previous minor from the download site. +
When such think happen the building of the corresponding local image will fail in Eskimo. +
The only solution to this is to update the target version in the file packages_dev/common/common.sh.
== Eskimo source distribution layout
The directory structure of the eskimo source distribution is as follows:
doc contains the source documentation in asciidoc formatlibs contains dependencies not available in standard maven repositoriespackages_dev contains the docker images development framework and packagespackages_distrib is the destination folder in which downloaded or built images are placedservices_setup contains the services installation framework and packagessrc contains the source files to build eskimotest_lab contains various tools to build VMs aimed at testing eskimo== Test Laboratory
The folder test_lab folder in the Eskimo CE project root folder contains a Vagrant framework used to create virtual
machines to test Eskimo.
The supported virtual machines are declared in the file VagrantFile in the variable nodes:
The defined VMs use different Operating Systems to test Eskimo’s installation on different OSes. +
One can edit this variable to declare one’s own VM’s as required.
An individual VM is then started with vargrant using VirtualBox as follows:
Vagrant can also start VM’s using libvirt / QEMU / kvm if the required libraries and provider are properly installed:
=== Integration tests
A script integration-test.sh is provided in test_lab/integration_test that is instrumental in building and testing
eskimo in depth. +
It enables to test rebuilding a full fledged Eskimo environment from scratch on a test platform of one single of
multiple node(s) (VMs) and test all its features (run all the zeppelin sample notebooks, ensure proper deployment of
all services, etc.). +
A specific command line flag in this script even enables to build a full Demo VM from scratch (See user guide on Demo
VM).
It’s usage is at follows:
For instance, the DemoVM is built using the following call:
Multiple dependencies are required on the machine where this integration-test.sh script is intended to run:
All of these needs to be available in the PATH. (Lower versions of these software components might work as well but
without any guarantee).
== Further information
Look at the following files for more information
readme.adoc in the folder packages_dev] for information about the components docker images building frameworkreadme.adoc in the folder services_setup] for information about the services installation and operation framework.[appendix]
== Copyright and License
Eskimo is Copyright 2019 - 2023 eskimo.sh - All rights reserved. +
Author : http://www.eskimo.sh
Eskimo is available under a dual licensing model : commercial and GNU AGPL. +
If you did not acquire a commercial licence for Eskimo, you can still use it and consider it free software under the
terms of the GNU Affero Public License. You can redistribute it and/or modify it under the terms of the GNU Affero
Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option)
any later version. +
Compliance to each and every aspect of the GNU Affero Public License is mandatory for users who did no acquire a
commercial license.
Eskimo is distributed as a free software under GNU AGPL in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Affero Public License for more details.
You should have received a copy of the GNU Affero Public License along with Eskimo. If not,
see https://www.gnu.org/licenses/ or write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
Boston, MA, 02110-1301 USA.
You can be released from the requirements of the license by purchasing a commercial license. Buying such a
commercial license is mandatory as soon as :
For more information, please contact eskimo.sh at https://www.eskimo.sh
The above copyright notice and this licensing notice shall be included in all copies or substantial portions of the
Software.