Outline Server, developed by Jigsaw. The Outline Server is a proxy server that runs a Shadowsocks instance and provides a REST API for access key management. https://getoutline.org/
Find a file
Jonathan Cohen dafa1f5bc2
Allow users to change the hostname for new access keys (#565)
* Create an endpoint to change the server's hostname.

This is the backend only, and does not check if the hostname exists or points to a Shadowbox
server.  Users have the responsibility of double checking that the
hostname and DNS if necessary are properly set up and specified.

* Updates

Changes validation to use regex.

Gets rid of a bad test

Uses paramtereized tests for better test output

Adds setHostname to AccessKeyRepository and calls it in the endpoint

* Integration tests

Also fixes missing dependency on ip-regex

* Remove validator from lockfile

* Ensure we correctly use Python 2 for Cloud SDK

* API for the new endpoint to change the server hostname. (#534)

* API for the new endpoint to change the server hostname.

I am sending it up now so we can iterate on it while I'm busy
implementing the endpoint.  I will merge it into the feature branch when
it's done as opposed as into master.

* API for the new endpoint to change the server hostname.

I am sending it up now so we can iterate on it while I'm busy
implementing the endpoint.  I will merge it into the feature branch when
it's done as opposed as into master.

* Remove 404 and 406 error states.  Checking for a functioning Shadowbox
server requires knowledge of the API prefix, which creates layering
challenges.  The UI will be able to make these checks, but users who
call the API directly will have to be careful not to put in an incorrect
hostname.  The consequence of this would be that newly made access keys
won't work.

* Clarify API, change name, add validation errors

* Clarify description

* Remove the API response corresponding to a failed check against the new
hostname that it points to the same server

* Respond to review comments

* Rename endpoint and related functions

From the server's perspective, this does change the hostname for
existing access keys since on server load time (in the Server Manager)
we recalculate access key hostnames from the key repository.

* Respond to review comments

* Use version 1.2.0 since data limits is taking 1.1.0

* Fix update_mmdb and the Docker image to work on Alpine (#559)

* Fix the for loop in update_mmdb to be `dash` compatible

for x in {a..b} is bash-only, and alpine's default shell is `dash`.

* Fix other issues related to using musl/Busybox/dash

* Use `[[` instead of `((` for the branch
* Make sure to update to GNU coreutils in the image in order to use
`date --date`

* Respond to review comments

* UI for setting the hostname for access key invitations (#553)

* Increase Shadowbox version for data limits experiment (#544)

* Return the port for new access keys from GET /server.  (#552)

This was accidentally removed in #537.

* UI for setting the hostname for access key invitations

This also includes some random typo fixes and QOL improvements.

To test, you can use

    export SB_IMAGE=cohenjongoogle/shadowbox:latest
    yarn do shadowbox/docker/run

then in a different terminal

    yarn do server_manager/electron_app/run

and add the server

* Add back in new files

Also fixes a backward compatibility issues for servers which don't
support changing the hostname

* Respond to review comments

* Respond to review comments

* Respond to review comments

* Respond to review comments

* Use DB-IP's Geolocation database instead of MaxMind's (#557)

* Use DB-IP's free IP-to-Country database instead of MaxMind's

This avoids the necesseity to either (1) bundle MaxMind's database in
our docker image or (2) require server admins to individually create
MaxMind accounts due to the new requirements for using GeoLite2
databases.  See https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-using-geolite2-databases/

* Add attribution to DB-IP for the GeoIP database

* Make sure that we deal with lag between the changeover of the current
month and release of new versions of the DB-IP database

* Respond to review comments

* Attribute DB-IP in the update script

* Respond to review comments

* Respond to review comments

* Fix update_mmdb and the Docker image to work on Alpine (#559)

* Fix the for loop in update_mmdb to be `dash` compatible

for x in {a..b} is bash-only, and alpine's default shell is `dash`.

* Fix other issues related to using musl/Busybox/dash

* Use `[[` instead of `((` for the branch
* Make sure to update to GNU coreutils in the image in order to use
`date --date`

* Respond to review comments

* Update to Typescript 3.7 (#558)

See https://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-7.html for more information.  Among the nice new features are optional chainging and null coalescing, and assertion functions, as well as some additional safety checks in the compiler

* Create an endpoint to change the server's hostname.

This is the backend only, and does not check if the hostname exists or points to a Shadowbox
server.  Users have the responsibility of double checking that the
hostname and DNS if necessary are properly set up and specified.

* Updates

Changes validation to use regex.

Gets rid of a bad test

Uses paramtereized tests for better test output

Adds setHostname to AccessKeyRepository and calls it in the endpoint

* Integration tests

Also fixes missing dependency on ip-regex

* Remove validator from lockfile

* Ensure we correctly use Python 2 for Cloud SDK

* API for the new endpoint to change the server hostname. (#534)

* API for the new endpoint to change the server hostname.

I am sending it up now so we can iterate on it while I'm busy
implementing the endpoint.  I will merge it into the feature branch when
it's done as opposed as into master.

* API for the new endpoint to change the server hostname.

I am sending it up now so we can iterate on it while I'm busy
implementing the endpoint.  I will merge it into the feature branch when
it's done as opposed as into master.

* Remove 404 and 406 error states.  Checking for a functioning Shadowbox
server requires knowledge of the API prefix, which creates layering
challenges.  The UI will be able to make these checks, but users who
call the API directly will have to be careful not to put in an incorrect
hostname.  The consequence of this would be that newly made access keys
won't work.

* Clarify API, change name, add validation errors

* Clarify description

* Remove the API response corresponding to a failed check against the new
hostname that it points to the same server

* Respond to review comments

* Rename endpoint and related functions

From the server's perspective, this does change the hostname for
existing access keys since on server load time (in the Server Manager)
we recalculate access key hostnames from the key repository.

* Respond to review comments

* Use version 1.2.0 since data limits is taking 1.1.0

* Fix update_mmdb and the Docker image to work on Alpine (#559)

* Fix the for loop in update_mmdb to be `dash` compatible

for x in {a..b} is bash-only, and alpine's default shell is `dash`.

* Fix other issues related to using musl/Busybox/dash

* Use `[[` instead of `((` for the branch
* Make sure to update to GNU coreutils in the image in order to use
`date --date`

* Respond to review comments

* UI for setting the hostname for access key invitations

This also includes some random typo fixes and QOL improvements.

To test, you can use

    export SB_IMAGE=cohenjongoogle/shadowbox:latest
    yarn do shadowbox/docker/run

then in a different terminal

    yarn do server_manager/electron_app/run

and add the server

* Add back in new files

Also fixes a backward compatibility issues for servers which don't
support changing the hostname

* Respond to review comments

* Respond to review comments

* Respond to review comments

* Respond to review comments

* Fix messed up rebase

* merge

* Clear merge markers

Co-authored-by: alalamav <2132122+alalamav@users.noreply.github.com>

* Create an endpoint to change the server's hostname.

This is the backend only, and does not check if the hostname exists or points to a Shadowbox
server.  Users have the responsibility of double checking that the
hostname and DNS if necessary are properly set up and specified.

* Updates

Changes validation to use regex.

Gets rid of a bad test

Uses paramtereized tests for better test output

Adds setHostname to AccessKeyRepository and calls it in the endpoint

* Integration tests

Also fixes missing dependency on ip-regex

* Remove validator from lockfile

* Ensure we correctly use Python 2 for Cloud SDK

* API for the new endpoint to change the server hostname. (#534)

* API for the new endpoint to change the server hostname.

I am sending it up now so we can iterate on it while I'm busy
implementing the endpoint.  I will merge it into the feature branch when
it's done as opposed as into master.

* API for the new endpoint to change the server hostname.

I am sending it up now so we can iterate on it while I'm busy
implementing the endpoint.  I will merge it into the feature branch when
it's done as opposed as into master.

* Remove 404 and 406 error states.  Checking for a functioning Shadowbox
server requires knowledge of the API prefix, which creates layering
challenges.  The UI will be able to make these checks, but users who
call the API directly will have to be careful not to put in an incorrect
hostname.  The consequence of this would be that newly made access keys
won't work.

* Clarify API, change name, add validation errors

* Clarify description

* Remove the API response corresponding to a failed check against the new
hostname that it points to the same server

* Respond to review comments

* Rename endpoint and related functions

From the server's perspective, this does change the hostname for
existing access keys since on server load time (in the Server Manager)
we recalculate access key hostnames from the key repository.

* Respond to review comments

* Use version 1.2.0 since data limits is taking 1.1.0

* UI for setting the hostname for access key invitations (#553)

* Increase Shadowbox version for data limits experiment (#544)

* Return the port for new access keys from GET /server.  (#552)

This was accidentally removed in #537.

* UI for setting the hostname for access key invitations

This also includes some random typo fixes and QOL improvements.

To test, you can use

    export SB_IMAGE=cohenjongoogle/shadowbox:latest
    yarn do shadowbox/docker/run

then in a different terminal

    yarn do server_manager/electron_app/run

and add the server

* Add back in new files

Also fixes a backward compatibility issues for servers which don't
support changing the hostname

* Respond to review comments

* Respond to review comments

* Respond to review comments

* Respond to review comments

* Use DB-IP's Geolocation database instead of MaxMind's (#557)

* Use DB-IP's free IP-to-Country database instead of MaxMind's

This avoids the necesseity to either (1) bundle MaxMind's database in
our docker image or (2) require server admins to individually create
MaxMind accounts due to the new requirements for using GeoLite2
databases.  See https://blog.maxmind.com/2019/12/18/significant-changes-to-accessing-and-using-geolite2-databases/

* Add attribution to DB-IP for the GeoIP database

* Make sure that we deal with lag between the changeover of the current
month and release of new versions of the DB-IP database

* Respond to review comments

* Attribute DB-IP in the update script

* Respond to review comments

* Respond to review comments

* Fix update_mmdb and the Docker image to work on Alpine (#559)

* Fix the for loop in update_mmdb to be `dash` compatible

for x in {a..b} is bash-only, and alpine's default shell is `dash`.

* Fix other issues related to using musl/Busybox/dash

* Use `[[` instead of `((` for the branch
* Make sure to update to GNU coreutils in the image in order to use
`date --date`

* Respond to review comments

* Update to Typescript 3.7 (#558)

See https://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-7.html for more information.  Among the nice new features are optional chainging and null coalescing, and assertion functions, as well as some additional safety checks in the compiler

* Create an endpoint to change the server's hostname.

This is the backend only, and does not check if the hostname exists or points to a Shadowbox
server.  Users have the responsibility of double checking that the
hostname and DNS if necessary are properly set up and specified.

* Updates

Changes validation to use regex.

Gets rid of a bad test

Uses paramtereized tests for better test output

Adds setHostname to AccessKeyRepository and calls it in the endpoint

* Integration tests

Also fixes missing dependency on ip-regex

* Remove validator from lockfile

* Ensure we correctly use Python 2 for Cloud SDK

* API for the new endpoint to change the server hostname. (#534)

* API for the new endpoint to change the server hostname.

I am sending it up now so we can iterate on it while I'm busy
implementing the endpoint.  I will merge it into the feature branch when
it's done as opposed as into master.

* API for the new endpoint to change the server hostname.

I am sending it up now so we can iterate on it while I'm busy
implementing the endpoint.  I will merge it into the feature branch when
it's done as opposed as into master.

* Remove 404 and 406 error states.  Checking for a functioning Shadowbox
server requires knowledge of the API prefix, which creates layering
challenges.  The UI will be able to make these checks, but users who
call the API directly will have to be careful not to put in an incorrect
hostname.  The consequence of this would be that newly made access keys
won't work.

* Clarify API, change name, add validation errors

* Clarify description

* Remove the API response corresponding to a failed check against the new
hostname that it points to the same server

* Respond to review comments

* Rename endpoint and related functions

From the server's perspective, this does change the hostname for
existing access keys since on server load time (in the Server Manager)
we recalculate access key hostnames from the key repository.

* Respond to review comments

* Use version 1.2.0 since data limits is taking 1.1.0

* Fix update_mmdb and the Docker image to work on Alpine (#559)

* Fix the for loop in update_mmdb to be `dash` compatible

for x in {a..b} is bash-only, and alpine's default shell is `dash`.

* Fix other issues related to using musl/Busybox/dash

* Use `[[` instead of `((` for the branch
* Make sure to update to GNU coreutils in the image in order to use
`date --date`

* Respond to review comments

* UI for setting the hostname for access key invitations

This also includes some random typo fixes and QOL improvements.

To test, you can use

    export SB_IMAGE=cohenjongoogle/shadowbox:latest
    yarn do shadowbox/docker/run

then in a different terminal

    yarn do server_manager/electron_app/run

and add the server

* Add back in new files

Also fixes a backward compatibility issues for servers which don't
support changing the hostname

* Respond to review comments

* Respond to review comments

* Respond to review comments

* Respond to review comments

* Fix messed up rebase

* merge

* Clear merge markers

Co-authored-by: alalamav <2132122+alalamav@users.noreply.github.com>

Co-authored-by: alalamav <2132122+alalamav@users.noreply.github.com>
2020-01-30 15:22:57 -05:00
release Correct latest version (#151) 2018-06-13 15:38:47 -04:00
scripts i18n: mirror UI for RTL languages, detect user locale (#463) 2019-08-28 11:11:13 -04:00
src Allow users to change the hostname for new access keys (#565) 2020-01-30 15:22:57 -05:00
third_party/prometheus Use DB-IP's Geolocation database instead of MaxMind's (#557) 2020-01-16 18:08:29 -05:00
.clang-format import code 2018-03-14 17:52:39 -04:00
.dockerignore Add .dockerignore 2018-05-30 23:35:11 -04:00
.gitignore Allow users to change the hostname for new access keys (#565) 2020-01-30 15:22:57 -05:00
.travis.yml Specify macOS image in Travis (#532) 2019-11-07 12:14:09 -05:00
CONTRIBUTING.md import code 2018-03-14 17:52:39 -04:00
jasmine.json Fix test 2018-08-31 14:29:12 -04:00
LICENSE import code 2018-03-14 17:52:39 -04:00
macos-signing-certificate.p12.enc Sign macOS manager on Travis (#12) 2018-03-23 14:08:27 -04:00
package.json Update to Typescript 3.7 (#558) 2020-01-17 16:42:02 -05:00
README.md spelling: javascript 2018-11-25 17:13:58 -05:00
tsconfig.json import code 2018-03-14 17:52:39 -04:00
tslint.json Clear unused imports and variables (#473) 2019-08-21 14:21:30 -04:00
yarn.lock Allow users to change the hostname for new access keys (#565) 2020-01-30 15:22:57 -05:00

Outline Server

Build Status

This repository has all the code needed to create and manage Outline servers on DigitalOcean. An Outline server runs instances of Shadowsocks proxies and provides an API used by the Outline Manager application.

Go to https://getoutline.org for ready-to-use versions of the software.

Components

The system comprises the following components:

  • Outline Server: a proxy server that runs a Shadowsocks instance for each access key and a REST API to manage the access keys. The Outline Server runs in a Docker container in the host machine.

    See src/shadowbox

  • Outline Manager: an Electron application that can create Outline Servers on the cloud and talks to their access key management API to manage who has access to the server.

    See src/server_manager

  • Metrics Server: a REST service that the Outline Server talks to if the user opts-in to anonymous metrics sharing.

    See src/metrics_server

Code Prerequisites

In order to build and run the code, you need the following installed:

  • Node
  • Yarn
  • Wine, if you would like to generate binaries for Windows.

Then you need to install all the NPM package dependencies:

yarn

Note: If you are using root (not recommended on your dev machine, maybe in a container), you need to add { "allow_root": true } to your /root/.bowerrc file.

This project uses Yarn workspaces.

Build System

We have a very simple build system based on package.json scripts that are called using yarn and a thin wrapper for what we call build "actions".

We've defined a do package.json script that takes an action parameter:

yarn do $ACTION

This command will define a do_action() function and call ${ACTION}_action.sh, which must exist. The called action script can use do_action to call its dependencies. The $ACTION parameter is always resolved from the project root, regardless of the caller location.

The idea of do_action is to keep the build logic next to where the relevant code is. It also defines two environmental variables:

  • ROOT_DIR: the root directory of the project, as an absolute path.
  • BUILD_DIR: where the build output should go, as an absolute path.

Build output

Building creates the following directories under build/:

  • web_app/: The Manager web app.
    • static/: The standalone web app static files. This is what one deploys to a web server or runs with Electron.
  • electron_app/: The launcher desktop Electron app
    • static/: The Manager Electron app to run with the electron command-line
    • bundled/: The Electron app bundled to run standalone on each platform
    • packaged/: The Electron app bundles packaged as single files for distribution
  • invite_page: the Invite Page
    • static: The standalone static files to be deployed
  • shadowbox: The Proxy Server

The directories have subdirectories for intermediate output:

  • ts/: Autogenerated Typescript files
  • js/: The output from compiling Typescript code
  • browserified/: The output of browserifying the JavaScript code

To clean up:

yarn run clean