fix nav highlight and tooltip rendering

resolves #62

.github/workflows/ci.yml

@@ -0,0 +1,54 @@
+name: ci
+
+on:
+  push:
+    branches:
+      - "main"
+    tags:
+      - "v*.*.*"
+  pull_request:
+    branches:
+      - "main"
+
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: ${{ github.repository }}
+
+jobs:
+  build-and-push-image:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+
+      - name: Log in to the Container registry
+        uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
+        with:
+          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+          # generate Docker tags based on the following events/attributes
+          tags: |
+            type=ref,event=pr
+            type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'main') }}
+            type=semver,pattern={{version}}
+            type=semver,pattern={{major}}.{{minor}}
+            type=semver,pattern={{major}}
+
+      - name: Build and push Docker image
+        uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
+        with:
+          context: .
+          push: ${{ github.event_name != 'pull_request' }}
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}

README.md

@@ -65,28 +65,30 @@
       * Reports how much currency that characters posses
     * U-Score:
       * Reports how much U-Score that characters posses
-  * Analytics:
-    * Provide reporting to Developers to help better solve issues
-    * Disabled by default. Set `ALLOW_ANALYTICS` to true to enable.
 
 # Deployment
 
 > **NOTE: This tutorial assumes you have a working DLU server instance and**
-> **some knowledge of Linux**
+> **some knowledge of command line interfaces on your chosen platform**
+
+
+**It is highly recommended to setup a reverse proxy via Nginx or some other tool and use SSL to secure your Nexus Dashboard instance if you are going to be opening it up to any non-LANs**
+ * [How to setup Nginx](https://www.digitalocean.com/community/tutorials/how-to-configure-nginx-as-a-reverse-proxy-on-ubuntu-22-04)
+ * [How to use certbot for SSL](https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-22-04)
+
 ## Docker
 
 ```bash
-
 docker run -d \
     -e APP_SECRET_KEY='<secret_key>' \
     -e APP_DATABASE_URI='mysql+pymysql://<username>:<password>@<host>:<port>/<database>' \
     # you can include other optional Environment Variables from below like this
-    -e REQUIRE_PLAY_KEY=True
-    -p 8000:8000/tcp
-    -v /path/to/unpacked/client:/app/luclient:rw \
+    -e REQUIRE_PLAY_KEY=True \
+    -p 8000:8000/tcp \
+    -v /path/to/logs:/logs:rw /
+    -v /path/to/unpacked/client:/app/luclient:ro \
     -v /path/to/cachedir:/app/cache:rw \
-    aronwk/nexus-dashboard:latest
-
+    ghcr.io/darkflameuniverse/nexusdashboard:latest
 ```
 
  * `/app/luclient` must be mapped to the location of an unpacked client
@@ -103,37 +105,41 @@ Please Reference `app/settings_exmaple.py` to see all the variables
     * APP_DATABASE_URI (Must be provided)
   * Everything else is optional and has defaults
 
-## Manual
+## Manual Linux Installation
 
 Thanks to [HailStorm32](https://github.com/HailStorm32) for this manual install guide!
+
 ### Setting Up The Environment
-First you will want to install the following packages by executing the following commands
+First you will want to install the following packages by executing the following commands presuming you are on a Debian based system.
+
 `sudo apt-get update`
+
 `sudo apt-get install -y python3 python3-pip sqlite3 git unzip libmagickwand-dev`
 
 > *Note: If  you are having issues with installing `sqlite3`, change it to `sqlite`*
 
 <br>
-Next we will clone the repository. You can clone it anywhere, but for the purpose of this tutorial, we will be cloning it to the home directory.
+Next you will want to clone the repository. You can clone it anywhere, but for the purpose of this tutorial, we will be cloning it to the home directory.'
+<br></br>
 
-`cd` *make sure you are in the home directory*
+Run `cd ~` to ensure that you are currently in the home directory then run the following command to clone the repository into our home directory
 `git clone https://github.com/DarkflameUniverse/NexusDashboard.git`
 
-You should now have a directory called `NexusDashboard`
+You should now have a directory called `NexusDashboard` present in your home directory
 
 ### Setting up
 
 Rename the example settings file
 `cp ~/NexusDashboard/app/settings_example.py ~/NexusDashboard/app/settings.py`
 
-Now let's open the settings file we just created and configure some of the settings
-`vim ~/NexusDashboard/app/settings.py`
->*Feel free to use any text editor you are more comfortable with instead of vim*
+Now let's open the settings file we just created and configure some of the settings with nano as it is a simple text editor that is easy to use
+`nano ~/NexusDashboard/app/settings.py`
+>*Obviously you can replace this with a text editor of your choice, nano is just the most simple to use out of the ones available by default on most Linux distros*
 
 <br>
-Inside this file is where you can change certain settings like user registration, email support and other things. In this tutorial I will only be focusing on the bare minimum to get up and running, but feel free to adjust what you would like
+Inside this file is where you can change certain settings like user registration, email support and other things. In this tutorial we will only be focusing on the bare minimum to get up and running, but feel free to adjust what you would like to fit your needs.
 
->*Note: Enabling the email option will require further setup that is outside the scope of this tutorial*
+>*Note: There are options in here that are related to email registration and password recovery among other features however those require extra setup not covered by this tutorial*
 
 The two important settings to configure are `APP_SECRET_KEY` and `APP_DATABASE_URI`
 
@@ -163,55 +169,56 @@ Once you are done making the changes, save and close the file
 
 We will need the following folders from the client
 ```
-locale  (all of the files inside)
+locale
+└───locale.xml
 
 res
-|_BrickModels
-|_brickprimitives
-|_textures
-|_ui
-|_brickdb.zip
+├───BrickModels
+├───brickprimitives
+├───textures
+├───ui
+├───brickdb.zip
 ```
 Put the two folders in `~/NexusDashboard/app/luclient`
 
 Unzip the `brickdb.zip` in place
 `unzip brickdb.zip`
 
-Remove the `.zip` after you have unzipped it
+Remove the `.zip` file after you have unzipped it, you can do that with
 `rm brickdb.zip`
 
 In the `luclient` directory you should now have a file structure that looks like this
 ```
-local
-|_locale.xml
+locale
+└───locale.xml
 
 res
-|_BrickModels
-	|_...
-|_brickprimitives
-	|_...
-|_textures
-	|_...
-|_ui
-	|_...
-|_Assemblies
-	|_...
-|_Primitives
-	|_...
-|_Materials.xml
-|_info.xml
+├───BrickModels
+│   └─── ...
+├───brickprimitives
+│	└─── ...
+├───textures
+│	└─── ...
+├───ui
+│	└─── ...
+├───Assemblies
+│	└─── ...
+├───Primitives
+│	└─── ...
+├───Materials.xml
+└───info.xml
 ```
 
 We will also need to copy the `CDServer.sqlite` database file from the server to the `~/NexusDashboard/app/luclient/res` folder
 
-Once the file is moved over, you will need to rename it to `cdclient.sqlite`
+Once the file is moved over, you will need to rename it to `cdclient.sqlite`, this can be done with the following command
 ```bash
 mv ~/NexusDashboard/app/luclient/res/CDServer.sqlite ~/NexusDashboard/app/luclient/res/cdclient.sqlite
 ```
 
 
 ##### Remaining Setup
-Run the following commands one at a time
+To finish this, we will need to install the python dependencies and run the database migrations, simply run the following commands one at a time
 ```bash
 cd ~/NexusDashboard
 pip install -r requirements.txt
@@ -219,11 +226,109 @@ pip install gunicorn
 flask db upgrade
 ```
 ##### Running the site
-You can run the site with
+Once all of the above is complete, you can run the site with the command
 `gunicorn -b :8000 -w 4 wsgi:app`
 
+## Manual Windows Setup
+
+While a lot of the setup on Windows is the same a lot of it can be completed with GUI interfaces and requires installing things from websites instead of the command line.
+
+### Setting Up The Environment
+You need to install the following prerequisites:
+
+  * [Python 3.8](https://www.python.org/downloads/release/python-380/)
+  * [Git](https://git-scm.com/downloads)
+  * [ImageMagick](https://docs.wand-py.org/en/latest/guide/install.html#install-imagemagick-on-windows)
+  * [7-Zip](https://www.7-zip.org/download.html)
+
+Next you will need to clone the repository. You can clone it anywhere, but for the purpose of this tutorial, you will want to clone it to your desktop just for simplicity, it can be moved after.
+
+Open a command prompt and run `cd Desktop` (The command line should place you in your Home directory be default) to ensure that you are currently in the desktop directory then run the following command to clone the repository into our desktop directory
+
+Run the following command to clone the repository `git clone https://github.com/DarkflameUniverse/NexusDashboard.git`
+
+You should now have a directory called `NexusDashboard` present on your desktop.
+
+### Setting up
+Now that we have the repository cloned you need to rename the example settings file, you can perform this manually in the GUI or you can use the command line, to do the latter run the following commands
+  * `cd NexusDashboard\app`
+  * `copy settings_example.py settings.py`
+
+Now let's open the settings file we just created and configure some of the settings with the Windows default notepad.
+* `notepad settings.py`
+
+Inside this file is where you can change certain settings like user registration, email support and other things. In this tutorial we will only be focusing on the bare minimum to get up and running, but feel free to adjust what you would like to fit your needs.
+
+> *Note: There are options in here that are related to email registration and password recovery among other features however those require extra setup not covered by this tutorial*
+
+The two important settings to configure are `APP_SECRET_KEY` and `APP_DATABASE_URI`
+
+For `APP_SECRET_KEY` you can just fill in any random 32 character string and for `APP_DATABASE_URI` you will need to fill in a connection string to your database. The connection string will look similar to this. You will need to fill in your own information for the username, password, host, port and database name.
+```
+APP_DATABASE_URI = "mysql+pymysql://<username>:<password>@<host>:<port>/<database>"
+```
+and the rest of the file can be left at the default values other than the `APP_SECRET_KEY` which you will need to fill in with random characters.
+
+Once you are done making the changes, save and close the file
+
+##### Client related files
+We will need the following folders from the client
+```
+locale
+└───locale.xml
+
+res
+├───BrickModels
+├───brickprimitives
+├───textures
+├───ui
+└───brickdb.zip
+```
+Put the two folders in `Desktop/NexusDashboard/app/luclient`
+
+Unzip the `brickdb.zip` in place using 7-Zip, you can do this by right clicking the file and selecting `7-Zip > Extract Here`.
+
+After doing this you can remove the `.zip`, simply delete the file.
+
+In the `luclient` directory you should now have a file structure that looks like this
+```
+locale
+└───locale.xml
+
+res
+├───BrickModels
+│   └─── ...
+├───brickprimitives
+│	└─── ...
+├───textures
+│	└─── ...
+├───ui
+│	└─── ...
+├───Assemblies
+│	└─── ...
+├───Primitives
+│	└─── ...
+├───Materials.xml
+└───info.xml
+```
+
+We will also need to copy the `CDServer.sqlite` database file from the server to the `Desktop/NexusDashboard/app/luclient/res` folder
+
+Once the file is moved over, you will need to rename it to `cdclient.sqlite`, this can be done by right clicking the file and selecting `Rename` and then changing the name to `cdclient.sqlite`
+
+##### Remaining Setup
+To finish this, we will need to install the python dependencies and run the database migrations, simply run the following commands one at a time in the root directory of the site, if you are not in the root directory you can run `cd Desktop/NexusDashboard` to get there (assuming you have opened a new terminal window)
+```bat
+pip install -r requirements.txt
+flask db upgrade
+```
+
+##### Running the site
+Once all of the above is complete, you can run the site with the command
+`flask run` however bare in mind that this is a development version of the site, at the moment running a production version of the site on Windows is not supported.
+
 # Development
 
-Please use [Editor Config](https://editorconfig.org/)
+Please use [Editor Config](https://editorconfig.org/) to maintain a consistent coding style between different editors and different contributors.
 
-  * `flask run` to run local dev server
+  * `python3 -m flask run` to run a local dev server

app/__init__.py

@@ -1,5 +1,5 @@
 import os
-from flask import Flask, url_for, redirect
+from flask import Flask, url_for, g, redirect
 from functools import wraps
 from flask_assets import Environment
 from webassets import Bundle
@@ -11,6 +11,7 @@ from flask_user import user_registered, current_user, user_logged_in
 from flask_wtf.csrf import CSRFProtect
 from flask_apscheduler import APScheduler
 from app.luclient import register_luclient_jinja_helpers
+import pathlib
 
 from app.commands import (
     init_db,
@@ -18,17 +19,15 @@ from app.commands import (
     load_property,
     gen_image_cache,
     gen_model_cache,
-    fix_clone_ids,
-    parse_lucache,
-    makeup_unlisted_objects,
-    gen_new_locales,
-    xref_scripts
+    fix_clone_ids
 )
 from app.models import Account, AccountInvitation, AuditLog
 
 import logging
 from logging.handlers import RotatingFileHandler
 
+from werkzeug.exceptions import HTTPException
+
 # Instantiate Flask extensions
 csrf_protect = CSRFProtect()
 scheduler = APScheduler()
@@ -78,6 +77,17 @@ def create_app():
     def debug(text):
         print(text)
 
+    @app.teardown_appcontext
+    def close_connection(exception):
+        cdclient = getattr(g, '_cdclient', None)
+        if cdclient is not None:
+            cdclient.close()
+
+    @app.template_filter()
+    def numberFormat(value):
+        return format(int(value), ',d')
+
+
     # add the commands to flask cli
     app.cli.add_command(init_db)
     app.cli.add_command(init_accounts)
@@ -85,10 +95,6 @@ def create_app():
     app.cli.add_command(gen_image_cache)
     app.cli.add_command(gen_model_cache)
     app.cli.add_command(fix_clone_ids)
-    app.cli.add_command(parse_lucache)
-    app.cli.add_command(makeup_unlisted_objects)
-    app.cli.add_command(gen_new_locales)
-    app.cli.add_command(xref_scripts)
 
     register_logging(app)
     register_settings(app)
@@ -96,6 +102,19 @@ def create_app():
     register_blueprints(app)
     register_luclient_jinja_helpers(app)
 
+    # Extract the brickdb if it's not already extracted
+    materials = pathlib.Path(f'{app.config["CACHE_LOCATION"]}Materials.xml')
+    if not materials.is_file():
+        # unzip the brickdb, and remove the import after
+        from zipfile import ZipFile
+        with ZipFile(f"{app.config['CLIENT_LOCATION']}res/brickdb.zip","r") as zip_ref:
+            zip_ref.extractall(app.config["CACHE_LOCATION"])
+        del ZipFile
+        # copy over the brick primitives, and remove the import after
+        from shutil import copytree
+        copytree(f"{app.config['CLIENT_LOCATION']}res/brickprimitives", f"{app.config['CACHE_LOCATION']}brickprimitives")
+        del copytree
+
     return app
 
 
@@ -120,7 +139,7 @@ def register_extensions(app):
 
     assets = Environment(app)
     assets.url = app.static_url_path
-    scss = Bundle('scss/site.scss', filters='libsass', output='site.css')
+    scss = Bundle('scss/site.scss', filters='libsass', output='css/site.css')
     assets.register('scss_all', scss)
 
 
@@ -159,7 +178,7 @@ def register_blueprints(app):
 
 def register_logging(app):
     # file logger
-    file_handler = RotatingFileHandler('nexus_dashboard.log', maxBytes=1024 * 1024 * 100, backupCount=20)
+    file_handler = RotatingFileHandler('logs/nexus_dashboard.log', maxBytes=1024 * 1024 * 100, backupCount=20)
     file_handler.setLevel(logging.INFO)
     formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
     file_handler.setFormatter(formatter)
@@ -194,9 +213,6 @@ def register_settings(app):
         'APP_DATABASE_URI',
         app.config['APP_DATABASE_URI']
     )
-    app.config['SQLALCHEMY_BINDS'] = {
-        'cdclient': 'sqlite:///luclient/res/cdclient.sqlite'
-    }
 
     # try to get overides, otherwise just use what we have already
     app.config['USER_ENABLE_REGISTER'] = os.getenv(
@@ -223,17 +239,23 @@ def register_settings(app):
         'USER_REQUIRE_INVITATION',
         app.config['USER_REQUIRE_INVITATION']
     )
-    app.config['ALLOW_ANALYTICS'] = os.getenv(
-        'ALLOW_ANALYTICS',
-        app.config['ALLOW_ANALYTICS']
-    )
+    app.config['SQLALCHEMY_ENGINE_OPTIONS'] = {
+        "pool_pre_ping": True,
+        "pool_size": 10,
+        "max_overflow": 2,
+        "pool_recycle": 300,
+        "pool_pre_ping": True,
+        "pool_use_lifo": True
+    }
     app.config['MAIL_SERVER'] = os.getenv(
         'MAIL_SERVER',
         app.config['MAIL_SERVER']
     )
-    app.config['MAIL_PORT'] = os.getenv(
-        'MAIL_USE_SSL',
+    app.config['MAIL_PORT'] = int(
+        os.getenv(
+        'MAIL_PORT',
         app.config['MAIL_PORT']
+        )
     )
     app.config['MAIL_USE_SSL'] = os.getenv(
         'MAIL_USE_SSL',
@@ -260,6 +282,35 @@ def register_settings(app):
         app.config['USER_EMAIL_SENDER_EMAIL']
     )
 
+    if "ENABLE_CHAR_XML_UPLOAD" not in app.config:
+        app.config['ENABLE_CHAR_XML_UPLOAD'] = False
+    app.config['ENABLE_CHAR_XML_UPLOAD'] = os.getenv(
+        'ENABLE_CHAR_XML_UPLOAD',
+        app.config['ENABLE_CHAR_XML_UPLOAD']
+    )
+
+    if "CLIENT_LOCATION" not in app.config:
+        app.config['CLIENT_LOCATION'] = 'app/luclient/'
+    app.config['CLIENT_LOCATION'] = os.getenv(
+        'CLIENT_LOCATION',
+        app.config['CLIENT_LOCATION']
+    )
+
+    if "CD_SQLITE_LOCATION" not in app.config:
+        app.config['CD_SQLITE_LOCATION'] = 'app/luclient/res/'
+    app.config['CD_SQLITE_LOCATION'] = os.getenv(
+        'CD_SQLITE_LOCATION',
+        app.config['CD_SQLITE_LOCATION']
+    )
+
+    if "CACHE_LOCATION" not in app.config:
+        app.config['CACHE_LOCATION'] = 'app/cache/'
+    app.config['CACHE_LOCATION'] = os.getenv(
+        'CACHE_LOCATION',
+        app.config['CACHE_LOCATION']
+    )
+
+
 
 def gm_level(gm_level):
     """Decorator for handling permissions based on the user's GM Level

app/accounts.py

@@ -1,7 +1,9 @@
 from flask import render_template, Blueprint, redirect, url_for, request, current_app, flash
 from flask_user import login_required, current_user
 from datatables import ColumnDT, DataTables
+import bcrypt
 import datetime
+import secrets
 from app.models import (
     Account,
     CharacterInfo,
@@ -14,11 +16,13 @@ from app.models import (
     AuditLog,
     BugReport,
     AccountInvitation,
-    db
+    db,
+    Friends
 )
 from app.schemas import AccountSchema
 from app import gm_level, log_audit
 from app.forms import EditGMLevelForm, EditEmailForm
+from sqlalchemy import or_
 
 accounts_blueprint = Blueprint('accounts', __name__)
 
@@ -150,10 +154,14 @@ def delete(id):
     message = f"Deleted Account ({account.id}){account.username}"
     chars = CharacterInfo.query.filter(CharacterInfo.account_id == id).all()
     for char in chars:
-        activities = ActivityLog.query.filter(ActivityLog.character_id == char.id).all()
+        activities = ActivityLog.query.filter(
+            ActivityLog.character_id == char.id
+        ).all()
         for activity in activities:
             activity.delete()
-        lb_entries = Leaderboard.query.filter(Leaderboard.character_id == char.id).all()
+        lb_entries = Leaderboard.query.filter(
+            Leaderboard.character_id == char.id
+        ).all()
         for lb_entry in lb_entries:
             lb_entry.delete()
         mails = Mail.query.filter(Mail.receiver_id == char.id).all()
@@ -161,12 +169,19 @@ def delete(id):
             mail.delete()
         props = Property.query.filter(Property.owner_id == char.id).all()
         for prop in props:
-            prop_contents = PropertyContent.query.filter(PropertyContent.property_id == prop.id).all()
+            prop_contents = PropertyContent.query.filter(
+                PropertyContent.property_id == prop.id
+            ).all()
             for prop_content in prop_contents:
                 if prop_content.lot == "14":
                     UGC.query.filter(UGC.id == prop.ugc_id).first().delete()
                 prop_content.delete()
             prop.delete()
+        friends = Friends.query.filter(
+            or_(Friends.player_id == char.id, Friends.friend_id == char.id)
+        ).all()
+        for friend in friends:
+            friend.delete()
         char.delete()
     # This is for GM stuff, it will be permnently delete logs
     bugs = BugReport.query.filter(BugReport.resoleved_by_id == id).all()
@@ -175,7 +190,8 @@ def delete(id):
     audits = AuditLog.query.filter(AuditLog.account_id == id).all()
     for audit in audits:
         audit.delete()
-    invites = AccountInvitation.query.filter(AccountInvitation.invited_by_user_id == id).all()
+    invites = AccountInvitation.query.filter(
+        AccountInvitation.invited_by_user_id == id).all()
     for invite in invites:
         invite.delete()
     account.delete()
@@ -184,6 +200,27 @@ def delete(id):
     return redirect(url_for("main.index"))
 
 
+@accounts_blueprint.route('/pass_reset/<id>', methods=['GET', 'POST'])
+@login_required
+@gm_level(9)
+def pass_reset(id):
+    # get the account
+    account = Account.query.filter(Account.id == id).first()
+    # make a random pass of length 12 using secrets
+    raw_pass = secrets.token_urlsafe(12)
+    # generate the hash
+    salt = bcrypt.gensalt()
+    hashed = bcrypt.hashpw(str.encode(raw_pass), salt)
+    # save the has
+    account.password = hashed
+    account.save()
+    # display for the admin to get and log that the action was done
+    flash(f"Set password for account {account.username} to {raw_pass}", "success")
+    log_audit(f"Reset password for {account.username}")
+
+    return redirect(request.referrer if request.referrer else url_for("main.index"))
+
+
 @accounts_blueprint.route('/get', methods=['GET'])
 @login_required
 @gm_level(3)

app/characters.py

@@ -1,14 +1,15 @@
-from flask import render_template, Blueprint, redirect, url_for, request, abort, flash, make_response
+from flask import render_template, Blueprint, redirect, url_for, request, abort, flash, make_response, current_app
 from flask_user import login_required, current_user
 from datatables import ColumnDT, DataTables
 import time
 from app.models import CharacterInfo, CharacterXML, Account, db
 from app.schemas import CharacterInfoSchema
-from app.forms import RescueForm
+from app.forms import RescueForm, CharXMLUploadForm
 from app import gm_level, log_audit
 from app.luclient import translate_from_locale
 import xmltodict
 import xml.etree.ElementTree as ET
+import json
 
 
 character_blueprint = Blueprint('characters', __name__)
@@ -78,7 +79,7 @@ def view(id):
     character_json = xmltodict.parse(
         CharacterXML.query.filter(
             CharacterXML.id == id
-        ).first().xml_data,
+        ).first().xml_data.replace("\"stt=", "\" stt="),
         attr_prefix="attr_"
     )
 
@@ -91,9 +92,10 @@ def view(id):
     # stupid fix for jinja parsing
     character_json["obj"]["inv"]["holdings"] = character_json["obj"]["inv"].pop("items")
     # sort by items slot index
-    for inv in character_json["obj"]["inv"]["holdings"]["in"]:
-        if "i" in inv.keys() and type(inv["i"]) == list:
-            inv["i"] = sorted(inv["i"], key=lambda i: int(i['attr_s']))
+    if type(character_json["obj"]["inv"]["holdings"]["in"]) == list:
+        for inv in character_json["obj"]["inv"]["holdings"]["in"]:
+                if "i" in inv.keys() and type(inv["i"]) == list:
+                    inv["i"] = sorted(inv["i"], key=lambda i: int(i['attr_s']))
 
     return render_template(
         'character/view.html.j2',
@@ -119,7 +121,7 @@ def view_xml(id):
 
     character_xml = CharacterXML.query.filter(
         CharacterXML.id == id
-    ).first().xml_data
+    ).first().xml_data.replace("\"stt=", "\" stt=")
 
     response = make_response(character_xml)
     response.headers.set('Content-Type', 'text/xml')
@@ -190,7 +192,7 @@ def rescue(id):
         CharacterXML.id == id
     ).first()
 
-    character_xml = ET.XML(character_data.xml_data)
+    character_xml = ET.XML(character_data.xml_data.replace("\"stt=", "\" stt="))
     for zone in character_xml.findall('.//r'):
         if int(zone.attrib["w"]) % 100 == 0:
             form.save_world.choices.append(
@@ -216,6 +218,30 @@ def rescue(id):
     return render_template("character/rescue.html.j2", form=form)
 
 
+@character_blueprint.route('/upload/<id>', methods=['GET', 'POST'])
+@login_required
+@gm_level(8)
+def upload(id):
+    if not current_app.config["ENABLE_CHAR_XML_UPLOAD"]:
+        flash("You must enable this setting to do this", "danger")
+        return redirect(url_for('characters.view', id=id))
+
+    form = CharXMLUploadForm()
+
+    character_data = CharacterXML.query.filter(
+        CharacterXML.id == id
+    ).first()
+
+    if form.validate_on_submit():
+        character_data.xml_data = form.char_xml.data
+        character_data.save()
+        flash("You accept all consequences from these actions", "danger")
+        log_audit(f"Updated {character_data.id}'s xml data")
+        return redirect(url_for('characters.view', id=id))
+    form.char_xml.data = character_data.xml_data
+    return render_template("character/upload.html.j2", form=form)
+
+
 @character_blueprint.route('/get/<status>', methods=['GET'])
 @login_required
 @gm_level(3)
@@ -234,7 +260,7 @@ def get(status):
     if status == "approved":
         query = db.session.query().select_from(CharacterInfo).join(Account).filter((CharacterInfo.pending_name == "") & (CharacterInfo.needs_rename == False))
     elif status == "unapproved":
-        query = db.session.query().select_from(CharacterInfo).join(Account).filter((CharacterInfo.pending_name != "") | (CharacterInfo.needs_rename == True))
+        query = db.session.query().select_from(CharacterInfo).join(Account).filter((CharacterInfo.pending_name != "") & (CharacterInfo.needs_rename == False))
     else:
         query = db.session.query().select_from(CharacterInfo).join(Account)
 

app/commands.py

@@ -15,17 +15,7 @@ from multiprocessing import Pool
 from functools import partial
 from sqlalchemy import func
 import time
-import csv
-import json
-from app.cdclient import (
-    ComponentsRegistry,
-    RenderComponent,
-    ItemComponent,
-    Objects,
-    ScriptComponent,
-)
 
-from app.luclient import translate_from_locale
 
 @click.command("init_db")
 @click.argument('drop_tables', nargs=1)
@@ -190,187 +180,6 @@ def load_property(zone, player):
             )
             new_prop_content.save()
 
-@click.command("parse_lucache")
-@with_appcontext
-def parse_lucache():
-    """Parses lucache csv file dump from nexus hq"""
-    unlisted_ids = [146, 147, 938, 1180, 1692, 1715, 1797, 1799, 1824, 1846, 1847, 1848, 1849, 1850, 1872, 1877, 1887, 1928, 1937, 1968, 1970, 1971, 1972, 1974, 1976, 1977, 1978, 1979, 1980, 1981, 1983, 1984, 2189, 2401, 2402, 2403, 2404, 2405, 2406, 2407, 2408, 2416, 2417, 2418, 2420, 2421, 2422, 2423, 2424, 2425, 2426, 2427, 2429, 2430, 2431, 2432, 2433, 2434, 2435, 2436, 2529, 2530, 2553, 2583, 2655, 2656, 2669, 2947, 2948, 3009, 3058, 3068, 3078, 3807, 3812, 3937, 4828, 4874, 4875, 4876, 4877, 4943, 4954, 5839, 5840, 6196, 6218, 6219, 6221, 6433, 6471, 6696, 6821, 6877, 6888, 6889, 6891, 6892, 6893, 6894, 6896, 6897, 6983, 7277, 7551, 7552, 7553, 7554, 7609, 7701, 7713, 7723, 7753, 7754, 7755, 7756, 7760, 7777, 7791, 7824, 7872, 8046, 8053, 8146, 9865, 9866, 9867, 9868, 10126, 10291, 10292, 10293, 10294, 10518, 10630, 10631, 10987, 11511, 11512, 11513, 11514, 11515, 11516, 11517, 11518, 11519, 11520, 11521, 11522, 11523, 11524, 11525, 12096, 12097, 12099, 12100, 12104, 12105, 12111, 12112, 12113, 12324, 12325, 12326, 12553, 12666, 12668, 12670, 12671, 12673, 12674, 12676, 12679, 12680, 12683, 12684, 12685, 12687, 12692, 12694, 12697, 12699, 12701, 12703, 12704, 12713, 12716, 12717, 12727, 12736, 12738, 12739, 12745, 12746, 12750, 12751, 12752, 12757, 12787, 12790, 12791, 12794, 12795, 12799, 12800, 12803, 12887, 12888, 12902, 12904, 12905, 12906, 12907, 12941, 13060, 13061, 13071, 13075, 13076, 13077, 13092, 13093, 13094, 13106, 13118, 13121, 13126, 13127, 13150, 13191, 13192, 13275, 13276, 13277, 13278, 13280, 13295, 13410, 13411, 13510, 13638, 13740, 13742, 13776, 13782, 13905, 13925, 13926, 13927, 13928, 13929, 13930, 13931, 13932, 13953, 13958, 13974, 13996, 13997, 13998, 13999, 14000, 14001, 14002, 14056, 14057, 14058, 14059, 14060, 14061, 14062, 14063, 14064, 14065, 14066, 14067, 14068, 14069, 14070, 14071, 14072, 14073, 14074, 14075, 14076, 14077, 14078, 14079, 14080, 14081, 14090, 14094, 14111, 14135, 14140, 14170, 14171, 14188, 14200, 14202, 14206, 14207, 14208, 14209, 14210, 14211, 14212, 14213, 14228, 14229, 14314, 14428, 14483, 14515, 14522, 14531, 14535, 14536, 14538, 14548, 14554, 14587, 14588, 14589, 14597, 14598, 14599, 14605, 14607, 14608, 14609, 14610, 14611, 14612, 14613, 14614, 14615, 14616, 14617, 14618, 14619, 14620, 14621, 14622, 14623, 14624, 14625, 14626, 14627, 14628, 14629, 14630, 14631, 14632, 14633, 14634, 14635, 14636, 14637, 14638, 14639, 14640, 14641, 14642, 14643, 14644, 14645, 14646, 14647, 14648, 14649, 14650, 14651, 14652, 14653, 14654, 14655, 14656, 14657, 14658, 14659, 14660, 14661, 14662, 14663, 14664, 14665, 14666, 14667, 14668, 14686, 14687, 14688, 14689, 14690, 14704, 14706, 14707, 14716, 14717, 14721, 14722, 14727, 14728, 14729, 14779, 14795, 14799, 14800, 14803, 14815, 14820, 14821, 14822, 14823, 14824, 14825, 14826, 14827, 14831, 14832, 14838, 14839, 15852, 15853, 15854, 15855, 15856, 15857, 15858, 15859, 15860, 15861, 15862, 15863, 15864, 15865, 15885, 15886, 15887, 15888, 15889, 15893, 15894, 15898, 15921, 15923, 15925, 15928, 15930, 15931, 15932, 15933, 15934, 15938, 15939, 15940, 15941, 15942, 15945, 15958, 15962, 15963, 15964, 15965, 15966, 15967, 15968, 15969, 15970, 15971, 15972, 15973, 15981, 15984, 15985, 15986, 15987, 15988, 15989, 15996, 15997, 15998, 15999, 16000, 16001, 16002, 16003, 16004, 16005, 16007, 16008, 16009, 16010, 16011, 16025, 16026, 16027, 16028, 16036, 16039, 16042, 16046, 16051, 16056, 16071, 16072, 16073, 16074, 16075, 16077, 16078, 16079, 16080, 16081, 16089, 16090, 16091, 16092, 16108, 16109, 16110, 16111, 16112, 16113, 16114, 16115, 16116, 16117, 16124, 16125, 16126, 16127, 16128, 16129, 16130, 16137, 16138, 16139, 16140, 16142, 16145, 16167, 16168, 16169, 16170, 16171, 16172, 16173, 16174, 16175, 16176, 16177, 16200, 16201, 16202, 16204, 16212, 16253, 16254, 16418, 16437, 16469, 16479, 16489, 16505, 16641, 16645, 16646, 16648, 16655, 16658, 16659, 16660, 16661, 16662, 16665, 16666, 16667, 16668, 16669, 16670, 16671, 16672, 16673, 16674, 16675, 16676, 16677, 16678, 16679, 16680, 16681, 16685, 16686, 16687, 16688, 16689, 16690, 16691, 16692, 16693, 16694, 16695, 16696, 16697, 16698, 16699, 16700, 16701, 16702, 16703, 16704, 16705, 16706, 16707, 16708, 16709, 16712, 16714, 16717, 16718, 16719, 16720, 16721, 16722, 16724, 16725, 16726, 16727, 16732, 16733, 16734, 16735] # noqa
-    with open("lucache.csv") as cache_file:
-        csv_reader = csv.reader(cache_file, delimiter=',')
-        line_count = 0
-        for row in csv_reader:
-            if row[0] == "id":
-                continue
-            if int(row[0]) in unlisted_ids:
-                json_data = json.loads(row[2])
-                components = ComponentsRegistry.query.filter(ComponentsRegistry.id == int(row[0])).all()
-                obj_type = "Environmental"
-                desc = json_data["Description"]
-                if desc in ["None", None, ""]:
-                    desc = row[1]
-                nametag = 0
-                npcTemplateID = "null"
-                for comp in components:
-                    if comp.component_type == 7:  # Item
-                        obj_type = "Smashable"
-                    if comp.component_type == 11:  # Item
-                        obj_type = "Loot"
-                    if comp.component_type == 35:  # minifig
-                        obj_type = "NPC"
-                        npcTemplateID = comp.component_id
-                        nametag = 1
-                    if comp.component_type == 42:  # b3
-                        obj_type = "Behavior"
-                    if comp.component_type == 60:  # base combat ai
-                        obj_type = "Enemy"
-                    if comp.component_type == 73:  # mission giver
-                        if obj_type != "NPC":
-                            obj_type = "Structure"
-                        desc = f"__MG__{desc}"
-                if "vendor" in row[1].lower():
-                    obj_type = "NPC"
-                print(f"""INSERT INTO "Objects" ("id","name","placeable","type","description","localize","npcTemplateID","displayName","interactionDistance","nametag","_internalNotes","locStatus","gate_version","HQ_valid") VALUES ("{row[0]}", "{row[1].replace("_", " ")}", 1, "{obj_type}", "{desc}", 1, {npcTemplateID} , "{json_data["DisplayName"]}", null , {nametag}, "Unlisted Object", 0, null, 1);""")
-                if obj_type in ["NPC", "Smashable", "Loot"]:
-                    print(f"""        <phrase id="Objects_{row[0]}_name">
-            <translation locale="en_US">{row[1]}</translation>
-            <translation locale="de_DE">TRASNSLATE UNLISTED</translation>
-            <translation locale="en_GB">{row[1]}</translation>
-        </phrase>
-        <phrase id="Objects_{row[0]}_description">
-            <translation locale="en_US">{desc}</translation>
-            <translation locale="de_DE">TRASNSLATE UNLISTED</translation>
-            <translation locale="en_GB">{desc}</translation>
-        </phrase>""")
-                # print(f'{row[0]}: {json_data["DisplayName"]}')
-                line_count += 1
-        # print(f'Processed {line_count} lines.')
-
-
-@click.command("makeup_unlisted_objects")
-@with_appcontext
-def makeup_unlisted_objects():
-    objs_left = []
-    for obj in objs_left:
-        obj_type = "Environmental"
-        nametag = 0
-        name = "Name Missing"
-        desc = "null"
-        npcTemplateID = "null"
-        components = ComponentsRegistry.query.filter(ComponentsRegistry.id == obj).all()
-        for comp in components:
-            if comp.component_type == 2:  # render
-                render = RenderComponent.query.filter(RenderComponent.id == comp.component_id).first()
-                if render is not None:
-
-                    if render.render_asset not in [None, ""]:
-                        name = render.render_asset.replace("_", " ").split('\\')[-1].split('/')[-1].split('.')[0].lower()
-                    if name == "Name Missing":
-                        if render.icon_asset not in [None, ""]:
-                            name = render.icon_asset.replace("_", " ").split('\\')[-1].split('/')[-1].split('.')[0].lower()
-                    name  = name.replace("env ", "").replace("obj ", "").replace("minifig accessory ", "").replace("", "").replace("mf ", "").replace("cre ", "")
-                    # print(f"{obj}: {name} :  {alt_name}")
-                    obj_type = "Smashable"
-                # else:
-                    # print(f"{obj}: No Render")
-            if comp.component_type == 7:  # destroyable
-                obj_type = "Smashable"
-            if comp.component_type == 11:  # Item
-                item = ItemComponent.query.filter(ItemComponent.id == comp.component_id).first()
-                if item.itemType == 24:
-                    obj_type = "Mount"
-                else:
-                    obj_type = "Loot"
-            if comp.component_type == 35:  # minifig
-                obj_type = "NPC"
-                npcTemplateID = comp.component_id
-                nametag = 1
-            if comp.component_type == 42:  # b3
-                obj_type = "Behavior"
-            if comp.component_type == 60:  # base combat ai
-                obj_type = "Enemy"
-            if comp.component_type == 73:  # mission giver
-                if obj_type != "NPC":
-                    obj_type = "Structure"
-                desc = f"__MG__{name}"
-        # print(f"""INSERT INTO "Objects" ("id","name","placeable","type","description","localize","npcTemplateID","displayName","interactionDistance","nametag","_internalNotes","locStatus","gate_version","HQ_valid") VALUES ("{obj}", "{name}", 1, "{obj_type}", "{desc}", 1, {npcTemplateID} , "{name}", null , {nametag}, "Unlisted Object", 0, null, 1);""")
-        if name != "Name Missing" and obj_type in ["Mount"]:
-            print(f"""        <phrase id="Objects_{obj}_name">
-            <translation locale="en_US">{name}</translation>
-            <translation locale="de_DE">TRASNSLATE UNLISTED</translation>
-            <translation locale="en_GB">{name}</translation>
-        </phrase>""")
-
-
-@click.command("gen_new_locales")
-@with_appcontext
-def gen_new_locales():
-    objects = Objects.query.order_by(Objects.id).all()
-    for obj in objects:
-        if obj.type == "Loot":
-            if obj.name not in ["Name Missing", None, "None"] and obj.name[:1] != "m":
-                name_to_trans = f"Object_{obj.id}_name"
-                name_transed = translate_from_locale(name_to_trans)
-                if name_to_trans == name_transed:
-                    print(f"""        <phrase id="Objects_{obj.id}_name">
-            <translation locale="en_US">{obj.name}</translation>
-            <translation locale="de_DE">TRASNSLATE OLD</translation>
-            <translation locale="en_GB">{obj.name}</translation>
-        </phrase>""")
-            if obj.description not in ["None", None, ""]:
-                description_to_trans = f"Object_{obj.id}_description"
-                description_transed = translate_from_locale(description_to_trans)
-                if description_to_trans == description_transed:
-                    print(f"""        <phrase id="Objects_{obj.id}_description">
-            <translation locale="en_US">{obj.description}</translation>
-            <translation locale="de_DE">TRASNSLATE OLD</translation>
-            <translation locale="en_GB">{obj.description}</translation>
-        </phrase>""")
-
-
-@click.command("xref_scripts")
-@with_appcontext
-def xref_scripts():
-    """cross refernce scripts dir with script component table"""
-    scripts = ScriptComponent.query.all()
-    base = 'app/luclient/res/'
-    server = 0
-    server_total = 0
-    client = 0
-    client_total = 0
-    server_used = 0
-    client_used = 0
-    used_total = 0
-    disk_scripts = [path for path in pathlib.Path('app/luclient/res/scripts').rglob("*.lua") if path.is_file()]
-
-    for script in scripts:
-        script_comps = ComponentsRegistry.query.filter(ComponentsRegistry.component_type == 5).filter(ComponentsRegistry.component_id == script.id).all()
-        if len(script_comps) > 0:
-            used_total += 1
-        if script.client_script_name not in [None, ""]:
-            cleaned_name = script.client_script_name.replace('\\', '/').lower()
-            client_script = pathlib.Path(f"{base}{cleaned_name}")
-            client_total += 1
-            if not client_script.is_file():
-                print(f"Missing Server Script: {client_script.as_posix()}")
-                client += 1
-                if len(script_comps) > 0:
-                    client_used += 1
-        if script.script_name not in [None, ""]:
-            cleaned_name = script.script_name.replace('\\', '/').lower()
-            server_script = pathlib.Path(f"{base}{cleaned_name}")
-            server_total += 1
-            if not server_script.is_file():
-                print(f"Missing Client Script: {server_script.as_posix()}")
-                server += 1
-                if len(script_comps) > 0:
-                    server_used += 1
-
-    print(f"Missing {server}/{server_total} server scripts")
-    print(f"Missing {client}/{client_total} client scripts")
-    print(f"Missing {server_used}/{used_total} used server scripts")
-    print(f"Missing {client_used}/{used_total} used client scripts")
-    print(f"Total cdclient scripts {server_total + client_total}\nTotal disk scripts {len(disk_scripts)}")
-
 
 @click.command("gen_image_cache")
 def gen_image_cache():

app/forms.py

@@ -75,10 +75,12 @@ class CustomRegisterForm(FlaskForm):
 
     password = PasswordField('Password', validators=[
         DataRequired(),
-        password_validator
+        password_validator,
+        validators.length(max=40, message="The maximum length of the password is 40 characters due to game client limitations")
     ])
     retype_password = PasswordField('Retype Password', validators=[
-        validators.EqualTo('password', message='Passwords did not match')
+        validators.EqualTo('password', message='Passwords did not match'),
+        validators.length(max=40, message="The maximum length of the password is 40 characters due to game client limitations")
     ])
 
     invite_token = HiddenField('Token')
@@ -209,3 +211,13 @@ class RejectPropertyForm(FlaskForm):
     )
 
     submit = SubmitField('Submit')
+
+
+class CharXMLUploadForm(FlaskForm):
+    char_xml = StringField(
+        'Paste minified charxml here:',
+        widget=TextArea(),
+        validators=[validators.DataRequired()]
+    )
+
+    submit = SubmitField('Submit')

app/log.py

@@ -25,7 +25,7 @@ def command():
 @log_blueprint.route('/system')
 @gm_level(8)
 def system():
-    with open('nexus_dashboard.log', 'r') as file:
+    with open('logs/nexus_dashboard.log', 'r') as file:
         logs = '</br>'.join(file.read().split('\n')[-100:])
     return render_template('logs/system.html.j2', logs=logs)
 

app/luclient.py

@@ -5,21 +5,11 @@ from flask import (
     redirect,
     url_for,
     make_response,
-    abort
+    abort,
+    current_app
 )
 from flask_user import login_required
 from app.models import CharacterInfo
-from app.cdclient import (
-    Objects,
-    Icons,
-    ItemSets,
-    ComponentsRegistry,
-    ComponentType,
-    RenderComponent,
-    ItemComponent,
-    ObjectSkills,
-    SkillBehavior
-)
 import glob
 import os
 from wand import image
@@ -27,8 +17,8 @@ from wand.exceptions import BlobError as BE
 import pathlib
 import json
 
+import sqlite3
 import xml.etree.ElementTree as ET
-from sqlalchemy import or_
 
 luclient_blueprint = Blueprint('luclient', __name__)
 locale = {}
@@ -43,7 +33,7 @@ def get_dds_as_png(filename):
     cache = f'cache/{filename.split(".")[0]}.png'
 
     if not os.path.exists(cache):
-        root = 'app/luclient/res/'
+        root = f"{current_app.config['CLIENT_LOCATION']}res/"
 
         path = glob.glob(
             root + f'**/{filename}',
@@ -52,7 +42,7 @@ def get_dds_as_png(filename):
 
         with image.Image(filename=path) as img:
             img.compression = "no"
-            img.save(filename='app/cache/' + filename.split('.')[0] + '.png')
+            img.save(filename=current_app.config["CACHE_LOCATION"] + filename.split('.')[0] + '.png')
 
     return send_file(cache)
 
@@ -63,7 +53,7 @@ def get_dds(filename):
     if filename.split('.')[-1] != 'dds':
         return 404
 
-    root = 'app/luclient/res/'
+    root = f"{current_app.config['CLIENT_LOCATION']}res/"
 
     dds = glob.glob(
         root + f'**/{filename}',
@@ -76,24 +66,37 @@ def get_dds(filename):
 @luclient_blueprint.route('/get_icon_lot/<id>')
 @login_required
 def get_icon_lot(id):
-    icon_path = RenderComponent.query.filter(
-        RenderComponent.id == ComponentsRegistry.query.filter(
-            ComponentsRegistry.component_type == ComponentType.COMPONENT_TYPE_RENDER
-        ).filter(ComponentsRegistry.id == id).first().component_id
-    ).first().icon_asset
-
-    if icon_path:
-        icon_path = icon_path.replace("..\\", "").replace("\\", "/")
+    if id is None:
+        redirect(url_for('luclient.unknown'))
+    render_component_id = query_cdclient(
+        'select component_id from ComponentsRegistry where component_type = 2 and id = ?',
+        [id],
+        one=True
+    )
+    if render_component_id is not None:
+        render_component_id = render_component_id[0]
     else:
         return redirect(url_for('luclient.unknown'))
 
-    cache = f'app/cache/{icon_path.split(".")[0]}.png'
+    # find the asset from rendercomponent given the  component id
+    filename = query_cdclient(
+        'select icon_asset from RenderComponent where id = ?',
+        [render_component_id],
+        one=True
+    )[0]
+
+    if filename:
+        filename = filename.replace("..\\", "").replace("\\", "/")
+    else:
+        return redirect(url_for('luclient.unknown'))
+
+    cache = f'{current_app.config["CACHE_LOCATION"]}{filename.split(".")[0]}.png'
 
     if not os.path.exists(cache):
-        root = 'app/luclient/res/'
+        root = f"{current_app.config['CLIENT_LOCATION']}res/"
         try:
             pathlib.Path(os.path.dirname(cache)).resolve().mkdir(parents=True, exist_ok=True)
-            with image.Image(filename=f'{root}{icon_path}'.lower()) as img:
+            with image.Image(filename=f'{root}{filename}'.lower()) as img:
                 img.compression = "no"
                 img.save(filename=cache)
         except BE:
@@ -106,14 +109,18 @@ def get_icon_lot(id):
 @login_required
 def get_icon_iconid(id):
 
-    filename = Icons.query.filter(Icons.IconID == id).first().IconPath
+    filename = query_cdclient(
+        'select IconPath from Icons where IconID = ?',
+        [id],
+        one=True
+    )[0]
 
     filename = filename.replace("..\\", "").replace("\\", "/")
 
-    cache = f'app/cache/{filename.split(".")[0]}.png'
+    cache = f'{current_app.config["CACHE_LOCATION"]}{filename.split(".")[0]}.png'
 
     if not os.path.exists(cache):
-        root = 'app/luclient/res/'
+        root = f"{current_app.config['CLIENT_LOCATION']}res/"
         try:
             pathlib.Path(os.path.dirname(cache)).resolve().mkdir(parents=True, exist_ok=True)
             with image.Image(filename=f'{root}{filename}'.lower()) as img:
@@ -131,14 +138,14 @@ def brick_list():
     brick_list = []
     if len(brick_list) == 0:
         suffixes = [".g", ".g1", ".g2", ".g3", ".xml"]
-        res = pathlib.Path('app/luclient/res/')
+        cache = pathlib.Path(f"{current_app.config['CACHE_LOCATION']}")
         # Load g files
-        for path in res.rglob("*.*"):
+        for path in cache.rglob("*.*"):
             if str(path.suffix) in suffixes:
                 brick_list.append(
                     {
                         "type": "file",
-                        "name": str(path.as_posix()).replace("app/luclient/res/", "")
+                        "name": str(path.as_posix()).replace(f"{current_app.config['CACHE_LOCATION']}", "")
                     }
                 )
     response = make_response(json.dumps(brick_list))
@@ -150,7 +157,7 @@ def brick_list():
 @luclient_blueprint.route('/ldddb/<path:req_path>')
 def dir_listing(req_path):
     # Joining the base and the requested path
-    rel_path = pathlib.Path(str(pathlib.Path(f'app/luclient/res/{req_path}').resolve()))
+    rel_path = pathlib.Path(str(pathlib.Path(f"{current_app.config['CACHE_LOCATION']}/{req_path}").resolve()))
     # Return 404 if path doesn't exist
     if not rel_path.exists():
         return abort(404)
@@ -167,10 +174,10 @@ def dir_listing(req_path):
 def unknown():
     filename = "textures/ui/inventory/unknown.dds"
 
-    cache = f'app/cache/{filename.split(".")[0]}.png'
+    cache = f'{current_app.config["CACHE_LOCATION"]}{filename.split(".")[0]}.png'
 
     if not os.path.exists(cache):
-        root = 'app/luclient/res/'
+        root = f"{current_app.config['CLIENT_LOCATION']}res/"
         try:
             pathlib.Path(os.path.dirname(cache)).resolve().mkdir(parents=True, exist_ok=True)
             with image.Image(filename=f'{root}{filename}'.lower()) as img:
@@ -182,6 +189,41 @@ def unknown():
     return send_file(pathlib.Path(cache).resolve())
 
 
+def get_cdclient():
+    """Connect to CDClient from file system Relative Path
+
+    Args:
+        None
+    """
+    cdclient = getattr(g, '_cdclient', None)
+    if cdclient is None:
+        path = pathlib.Path(f"{current_app.config['CD_SQLITE_LOCATION']}cdclient.sqlite")
+        if path.is_file():
+            cdclient = g._database = sqlite3.connect(f"{current_app.config['CD_SQLITE_LOCATION']}cdclient.sqlite")
+            return cdclient
+
+        path = pathlib.Path(f"{current_app.config['CD_SQLITE_LOCATION']}CDServer.sqlite")
+        if path.is_file():
+            cdclient = g._database = sqlite3.connect(f"{current_app.config['CD_SQLITE_LOCATION']}CDServer.sqlite")
+            return cdclient
+
+    return cdclient
+
+
+def query_cdclient(query, args=(), one=False):
+    """Run sql queries on CDClient
+
+    Args:
+        query   (string)    : SQL query
+        args    (list)      : List of args to place in query
+        one     (bool)      : Return only on result or all results
+    """
+    cur = get_cdclient().execute(query, args)
+    rv = cur.fetchall()
+    cur.close()
+    return (rv[0] if rv else None) if one else rv
+
+
 def translate_from_locale(trans_string):
     """Finds the string translation from locale.xml
 
@@ -196,7 +238,7 @@ def translate_from_locale(trans_string):
     locale_data = ""
 
     if not locale:
-        locale_path = "app/luclient/locale/locale.xml"
+        locale_path = f"{current_app.config['CLIENT_LOCATION']}locale/locale.xml"
 
         with open(locale_path, 'r') as file:
             locale_data = file.read()
@@ -220,11 +262,13 @@ def get_lot_name(lot_id):
         return "Missing"
     name = translate_from_locale(f'Objects_{lot_id}_name')
     if name == f'Objects_{lot_id}_name':
-        intermed = Objects.query.filter(Objects.id == lot_id).first()
+        intermed = query_cdclient(
+            'select * from Objects where id = ?',
+            [lot_id],
+            one=True
+        )
         if intermed:
-            name = intermed.displayName if (intermed.displayName != "None" or intermed.displayName != "" or intermed.displayName == None) else intermed.name
-    if not name:
-        name = f'Objects_{lot_id}_name'
+            name = intermed[7] if (intermed[7] != "None" and intermed[7] != "" and intermed[7] is None) else intermed[1]
     return name
 
 
@@ -252,6 +296,7 @@ def register_luclient_jinja_helpers(app):
 
     @app.template_filter('parse_lzid')
     def parse_lzid(lzid):
+        if not lzid: return [1000, 1000, 1000]
         return[
             (int(lzid) & ((1 << 16) - 1)),
             ((int(lzid) >> 16) & ((1 << 16) - 1)),
@@ -275,11 +320,21 @@ def register_luclient_jinja_helpers(app):
     def get_lot_rarity(lot_id):
         if not lot_id:
             return "Missing"
-        rarity = ItemComponent.query.filter(
-            ItemComponent.id == ComponentsRegistry.query.filter(
-                ComponentsRegistry.component_type == ComponentType.COMPONENT_TYPE_ITEM
-            ).filter(ComponentsRegistry.id == id).first().component_id
-        ).first().rarity
+        render_component_id = query_cdclient(
+            'select component_id from ComponentsRegistry where component_type = 11 and id = ?',
+            [lot_id],
+            one=True
+        )
+        if render_component_id:
+            render_component_id = render_component_id[0]
+
+        rarity = query_cdclient(
+            'select rarity from ItemComponent where id = ?',
+            [render_component_id],
+            one=True
+        )
+        if rarity:
+            rarity = rarity[0]
         return rarity
 
     @app.template_filter('get_lot_desc')
@@ -288,12 +343,15 @@ def register_luclient_jinja_helpers(app):
             return "Missing"
         desc = translate_from_locale(f'Objects_{lot_id}_description')
         if desc == f'Objects_{lot_id}_description':
-            desc = Objects.query.filter(Objects.id == lot_id).first()
-
+            desc = query_cdclient(
+                'select description from Objects where id = ?',
+                [lot_id],
+                one=True
+            )
             if desc in ("", None):
                 desc = None
             else:
-                desc = desc.description
+                desc = desc[0]
                 if desc in ("", None):
                     desc = None
         if desc:
@@ -304,14 +362,11 @@ def register_luclient_jinja_helpers(app):
     def check_if_in_set(lot_id):
         if not lot_id:
             return None
-        item_set = ItemSets.query.filter(
-            or_(
-                ItemSets.itemIDs.like(f'{lot_id}%'),
-                ItemSets.itemIDs.like(f'%, {lot_id}%'),
-                ItemSets.itemIDs.like(f'%,{lot_id}%')
-            )
-        ).first()
-
+        item_set = query_cdclient(
+            'select * from ItemSets where itemIDs like ? or itemIDs like ? or itemIDs like ?',
+            [f'{lot_id}%', f'%, {lot_id}%', f'%,{lot_id}%'],
+            one=True
+        )
         if item_set in ("", None):
             return None
         else:
@@ -321,19 +376,12 @@ def register_luclient_jinja_helpers(app):
     def get_lot_stats(lot_id):
         if not lot_id:
             return None
-        stats = SkillBehavior.query.with_entities(
-            SkillBehavior.imBonusUI,
-            SkillBehavior.lifeBonusUI,
-            SkillBehavior.armorBonusUI,
-            SkillBehavior.skillID,
-            SkillBehavior.skillIcon
-        ).filter(
-            SkillBehavior.skillID in ObjectSkills.query.with_entities(
-                ObjectSkills.skillID
-            ).filter(
-                ObjectSkills.objectTemplate == lot_id
-            ).all()
-        ).all()
+        stats = query_cdclient(
+            'SELECT imBonusUI, lifeBonusUI, armorBonusUI, skillID, skillIcon FROM SkillBehavior WHERE skillID IN (\
+                SELECT skillID FROM ObjectSkills WHERE objectTemplate=?\
+                )',
+            [lot_id]
+        )
 
         return consolidate_stats(stats)
 
@@ -341,22 +389,23 @@ def register_luclient_jinja_helpers(app):
     def get_set_stats(lot_id):
         if not lot_id:
             return "Missing"
-        stats = SkillBehavior.query.with_entities(
-            SkillBehavior.imBonusUI,
-            SkillBehavior.lifeBonusUI,
-            SkillBehavior.armorBonusUI,
-            SkillBehavior.skillID,
-            SkillBehavior.skillIcon
-        ).filter(
-            SkillBehavior.skillID == ObjectSkills.query.with_entities(
-                ObjectSkills.skillID
-            ).filter(
-                ObjectSkills.objectTemplate == lot_id
-            ).all()
-        ).all()
+        stats = query_cdclient(
+            'SELECT imBonusUI, lifeBonusUI, armorBonusUI, skillID, skillIcon FROM SkillBehavior WHERE skillID IN (\
+                SELECT skillID FROM ItemSetSkills WHERE SkillSetID=?\
+                )',
+            [lot_id]
+        )
 
         return consolidate_stats(stats)
 
+    @app.template_filter('query_cdclient')
+    def jinja_query_cdclient(query, items):
+        print(query, items)
+        return query_cdclient(
+            query,
+            items,
+            one=True
+        )[0]
 
     @app.template_filter('lu_translate')
     def lu_translate(to_translate):
@@ -365,18 +414,26 @@ def register_luclient_jinja_helpers(app):
 
 def consolidate_stats(stats):
 
-    if stats:
+    if len(stats) > 1:
         consolidated_stats = {"im": 0, "life": 0, "armor": 0, "skill": []}
         for stat in stats:
-            if stat.imBonusUI:
-                consolidated_stats["im"] += stat.imBonusUI
-            if stat.lifeBonusUI:
-                consolidated_stats["life"] += stat.lifeBonusUI
-            if stat.armorBonusUI:
-                consolidated_stats["armor"] += stat.armorBonusUI
-            if stat.skillID:
-                consolidated_stats["skill"].append([stat.skillID, stat.skillIcon])
+            if stat[0]:
+                consolidated_stats["im"] += stat[0]
+            if stat[1]:
+                consolidated_stats["life"] += stat[1]
+            if stat[2]:
+                consolidated_stats["armor"] += stat[2]
+            if stat[3]:
+                consolidated_stats["skill"].append([stat[3], stat[4]])
+
         stats = consolidated_stats
+    elif len(stats) == 1:
+        stats = {
+            "im": stats[0][0] if stats[0][0] else 0,
+            "life": stats[0][1] if stats[0][1] else 0,
+            "armor": stats[0][2] if stats[0][2] else 0,
+            "skill": [[stats[0][3], stats[0][4]]] if stats[0][3] else None,
+        }
     else:
         stats = None
     return stats

app/mail.py

@@ -3,8 +3,7 @@ from flask_user import login_required, current_user
 from app.models import Mail, CharacterInfo
 from app.forms import SendMailForm
 from app import gm_level, log_audit
-from app.luclient import translate_from_locale
-from app.cdclient import Objects
+from app.luclient import translate_from_locale, query_cdclient
 import time
 
 mail_blueprint = Blueprint('mail', __name__)
@@ -69,12 +68,14 @@ def send():
     for character in recipients:
         form.recipient.choices.append((character.id, character.name))
 
-    items = Objects.query.filter(Objects.type == "Loot").all()
+    items = query_cdclient(
+        'Select id, name, displayName from Objects where type = "Loot"'
+    )
 
     for item in items:
         name = translate_from_locale(f'Objects_{item[0]}_name')
-        if name == f'Objects_{item.id}_name':
-            name = (item.displayName if (item.displayName != "None" and item.displayName != "" and item.displayName is not None) else item.name)
+        if name == f'Objects_{item[0]}_name':
+            name = (item[2] if (item[2] != "None" and item[2] != "" and item[2] is not None) else item[1])
         form.attachment.choices.append(
             (
                 item[0],

app/main.py

@@ -4,6 +4,9 @@ from flask_user import current_user, login_required
 from app.models import Account, CharacterInfo, ActivityLog
 from app.schemas import AccountSchema, CharacterInfoSchema
 
+import datetime
+import time
+
 main_blueprint = Blueprint('main', __name__)
 
 account_schema = AccountSchema()
@@ -28,12 +31,16 @@ def index():
 @login_required
 def about():
     """About Page"""
-    mods = Account.query.filter(Account.gm_level > 0).order_by(Account.gm_level.desc()).all()
+    mods = Account.query.filter(Account.gm_level > 1).order_by(Account.gm_level.desc()).all()
     online = 0
-    chars = CharacterInfo.query.all()
+    users = []
+    zones = {}
+    twodaysago = time.mktime((datetime.datetime.now() - datetime.timedelta(days=2)).timetuple())
+    chars = CharacterInfo.query.filter(CharacterInfo.last_login >= twodaysago).all()
+
     for char in chars:
         last_log = ActivityLog.query.with_entities(
-            ActivityLog.activity
+            ActivityLog.activity, ActivityLog.map_id
         ).filter(
             ActivityLog.character_id == char.id
         ).order_by(ActivityLog.id.desc()).first()
@@ -41,8 +48,13 @@ def about():
         if last_log:
             if last_log[0] == 0:
                 online += 1
+                if current_user.gm_level >= 8: users.append([char.name, last_log[1]])
+                if str(last_log[1]) not in zones:
+                    zones[str(last_log[1])] = 1
+                else:
+                    zones[str(last_log[1])] += 1
 
-    return render_template('main/about.html.j2', mods=mods, online=online)
+    return render_template('main/about.html.j2', mods=mods, online=online, users=users, zones=zones)
 
 
 @main_blueprint.route('/favicon.ico')

app/play_keys.py

@@ -11,14 +11,14 @@ play_keys_blueprint = Blueprint('play_keys', __name__)
 # Key creation page
 @play_keys_blueprint.route('/', methods=['GET'])
 @login_required
-@gm_level(9)
+@gm_level(5)
 def index():
     return render_template('play_keys/index.html.j2')
 
 
 @play_keys_blueprint.route('/create/<count>/<uses>', methods=['GET'], defaults={'count': 1, 'uses': 1})
 @login_required
-@gm_level(9)
+@gm_level(5)
 def create(count=1, uses=1):
     key = PlayKey.create(count=count, uses=uses)
     log_audit(f"Created {count} Play Key(s) with {uses} uses!")
@@ -28,7 +28,7 @@ def create(count=1, uses=1):
 
 @play_keys_blueprint.route('/create/bulk', methods=('GET', 'POST'))
 @login_required
-@gm_level(9)
+@gm_level(5)
 def bulk_create():
     form = CreatePlayKeyForm()
     if form.validate_on_submit():
@@ -42,7 +42,7 @@ def bulk_create():
 
 @play_keys_blueprint.route('/delete/<id>', methods=('GET', 'POST'))
 @login_required
-@gm_level(9)
+@gm_level(5)
 def delete(id):
     key = PlayKey.query.filter(PlayKey.id == id).first()
     # associated_accounts = Account.query.filter(Account.play_key_id==id).all()
@@ -54,7 +54,7 @@ def delete(id):
 
 @play_keys_blueprint.route('/edit/<id>', methods=('GET', 'POST'))
 @login_required
-@gm_level(9)
+@gm_level(5)
 def edit(id):
     key = PlayKey.query.filter(PlayKey.id == id).first()
     form = EditPlayKeyForm()
@@ -81,7 +81,7 @@ def edit(id):
 
 @play_keys_blueprint.route('/view/<id>', methods=('GET', 'POST'))
 @login_required
-@gm_level(9)
+@gm_level(5)
 def view(id):
     key = PlayKey.query.filter(PlayKey.id == id).first()
     accounts = Account.query.filter(Account.play_key_id == id).all()
@@ -90,7 +90,7 @@ def view(id):
 
 @play_keys_blueprint.route('/get', methods=['GET'])
 @login_required
-@gm_level(9)
+@gm_level(5)
 def get():
     columns = [
         ColumnDT(PlayKey.id),

app/properties.py

@@ -13,10 +13,9 @@ from flask_user import login_required, current_user
 from datatables import ColumnDT, DataTables
 import time
 from app.models import Property, db, UGC, CharacterInfo, PropertyContent, Account, Mail
-from app.cdclient import ComponentsRegistry, ComponentType, RenderComponent
 from app.schemas import PropertySchema
 from app import gm_level, log_audit
-from app.cdclient import ZoneTable
+from app.luclient import query_cdclient
 from app.forms import RejectPropertyForm
 
 import zlib
@@ -50,9 +49,11 @@ def approve(id):
 
     if property_data.mod_approved:
         message = f"""Approved Property
-            {property_data.name if property_data.name else ZoneTable.query.filter(
-                ZoneTable.zoneID == property_data.zone_id
-            ).first().DisplayDescription}
+            {property_data.name if property_data.name else query_cdclient(
+                'select DisplayDescription from ZoneTable where zoneID = ?',
+                [property_data.zone_id],
+                one=True
+            )[0]}
             from {CharacterInfo.query.filter(CharacterInfo.id==property_data.owner_id).first().name}"""
         log_audit(message)
         flash(
@@ -61,9 +62,11 @@ def approve(id):
         )
     else:
         message = f"""Unapproved Property
-            {property_data.name if property_data.name else ZoneTable.query.filter(
-                ZoneTable.zoneID == property_data.zone_id
-            ).first().DisplayDescription}
+            {property_data.name if property_data.name else query_cdclient(
+                'select DisplayDescription from ZoneTable where zoneID = ?',
+                [property_data.zone_id],
+                one=True
+            )[0]}
             from {CharacterInfo.query.filter(CharacterInfo.id==property_data.owner_id).first().name}"""
         log_audit(message)
         flash(
@@ -97,9 +100,11 @@ def reject(id):
 
     if form.validate_on_submit():
         char_name = CharacterInfo.query.filter(CharacterInfo.id == property_data.owner_id).first().name
-        zone_name = ZoneTable.query.filter(
-            ZoneTable.zoneID == property_data.zone_id
-        ).first().DisplayDescription
+        zone_name = query_cdclient(
+            'select DisplayDescription from ZoneTable where zoneID = ?',
+            [property_data.zone_id],
+            one=True
+        )[0]
         property_data.mod_approved = False
         property_data.rejection_reason = form.rejection_reason.data
         message = f"""Rejected Property
@@ -205,7 +210,7 @@ def get(status="all"):
     rowTable = DataTables(params, query, columns)
 
     data = rowTable.output_result()
-
+    print(data)
     for property_data in data["data"]:
         id = property_data["0"]
 
@@ -246,9 +251,11 @@ def get(status="all"):
         """
 
         if property_data["4"] == "":
-            property_data["4"] = ZoneTable.query.filter(
-                ZoneTable.zoneID == property_data["13"]
-            ).first().DisplayDescription
+            property_data["4"] = query_cdclient(
+                'select DisplayDescription from ZoneTable where zoneID = ?',
+                [property_data["13"]],
+                one=True
+            )
 
         if property_data["6"] == 0:
             property_data["6"] = "Private"
@@ -265,9 +272,11 @@ def get(status="all"):
         else:
             property_data["7"] = '''<h2 class="far fa-check-square text-success"></h2>'''
 
-        property_data["13"] = ZoneTable.query.filter(
-            ZoneTable.zoneID == property_data["13"]
-        ).first().DisplayDescription
+        property_data["13"] = query_cdclient(
+            'select DisplayDescription from ZoneTable where zoneID = ?',
+            [property_data["13"]],
+            one=True
+        )
 
     return data
 
@@ -402,20 +411,37 @@ def download_model(id):
 
 def ugc(content):
     ugc_data = UGC.query.filter(UGC.id == content.ugc_id).first()
-    uncompressed_lxfml = zlib.decompress(ugc_data.lxfml)
+    uncompressed_lxfml = decompress(ugc_data.lxfml)
     response = make_response(uncompressed_lxfml)
     return response, ugc_data.filename
 
+def decompress(data):
+	assert data[:5] == b"sd0\x01\xff"
+	pos = 5
+	out = b""
+	while pos < len(data):
+		length = int.from_bytes(data[pos:pos+4], "little")
+		pos += 4
+		out += zlib.decompress(data[pos:pos+length])
+		pos += length
+	return out
 
 def prebuilt(content, file_format, lod):
     # translate LOT to component id
-    # we need to get a type of 2 because reasons
-    filename = RenderComponent.query.filter(
-        RenderComponent.id == ComponentsRegistry.query.filter(
-            ComponentsRegistry.component_type == ComponentType.COMPONENT_TYPE_RENDER
-        ).filter(ComponentsRegistry.id == id).first().component_id
-    ).first().render_asset
+    # we need to get a type of 2 for the render component to find the filename
+    render_component_id = query_cdclient(
+        'select component_id from ComponentsRegistry where component_type = 2 and id = ?',
+        [content.lot],
+        one=True
+    )[0]
+    # find the asset from rendercomponent given the component id
+    filename = query_cdclient(
+        'select render_asset from RenderComponent where id = ?',
+        [render_component_id],
+        one=True
+    )
 
+    # if we have a valie filename, coerce it
     if filename:
         filename = filename[0].split("\\\\")[-1].lower().split(".")[0]
         if "/" in filename:
@@ -423,25 +449,29 @@ def prebuilt(content, file_format, lod):
     else:
         return f"No filename for LOT {content.lot}"
 
-    lxfml = pathlib.Path(f'app/luclient/res/BrickModels/{filename.split(".")[0]}.lxfml')
+    # if we just want the lxfml, fine t and return it
+    lxfml = pathlib.Path(f'{current_app.config["CLIENT_LOCATION"]}res/BrickModels/{filename.split(".")[0]}.lxfml')
     if file_format == "lxfml":
 
         with open(lxfml, 'r') as file:
             lxfml_data = file.read()
         response = make_response(lxfml_data)
 
+    # else we handle getting the files for lddviewer
     elif file_format in ["obj", "mtl"]:
-        cache = pathlib.Path(f'app/cache/BrickModels/{filename}.lod{lod}.{file_format}')
+        # check to see if the file exists
+        cache = pathlib.Path(f'{current_app.config["CACHE_LOCATION"]}BrickModels/{filename}.lod{lod}.{file_format}')
         if not cache.is_file():
+            # if not make it an store it for later
             cache.parent.mkdir(parents=True, exist_ok=True)
             try:
                 ldd.main(str(lxfml.as_posix()), str(cache.with_suffix("").as_posix()), lod)  # convert to OBJ
             except Exception as e:
                 current_app.logger.error(f"ERROR on {cache}:\n {e}")
-
+        # then just read it
         with open(str(cache.as_posix()), 'r') as file:
             cache_data = file.read()
-
+        # and serve it
         response = make_response(cache_data)
 
     else:

app/pylddlib.py

@@ -7,6 +7,7 @@ import math
 import struct
 import zipfile
 from xml.dom import minidom
+from flask import current_app
 
 PRIMITIVEPATH = '/Primitives/'
 GEOMETRIEPATH = PRIMITIVEPATH
@@ -968,8 +969,8 @@ def main(lxf_filename, obj_filename, lod="2"):
     GEOMETRIEPATH = GEOMETRIEPATH + f"LOD{lod}/"
     converter = Converter()
     # print("Found DB folder. Will use this instead of db.lif!")
-    setDBFolderVars(dbfolderlocation="app/luclient/res/", lod=lod)
-    converter.LoadDBFolder(dbfolderlocation="app/luclient/res/")
+    setDBFolderVars(dbfolderlocation=f"{current_app.config['CACHE_LOCATION']}", lod=lod)
+    converter.LoadDBFolder(dbfolderlocation=f"{current_app.config['CACHE_LOCATION']}")
     converter.LoadScene(filename=lxf_filename)
     converter.Export(filename=obj_filename)
 

app/settings_example.py

@@ -7,14 +7,15 @@ APP_SYSTEM_ERROR_SUBJECT_LINE = APP_NAME + " system error"
 APP_SECRET_KEY = ""
 APP_DATABASE_URI = "mysql+pymysql://<username>:<password>@<host>:<port>/<database>"
 
+CLIENT_LOCATION = 'app/luclient/'
+CD_SQLITE_LOCATION = 'app/luclient/res/'
+CACHE_LOCATION = 'app/cache/'
+
 CONFIG_LINK = False
 CONFIG_LINK_TITLE = ""
 CONFIG_LINK_HREF = ""
 CONFIG_LINK_TEXT = ""
 
-# Send Analytics for Developers to better fix issues
-ALLOW_ANALYTICS = False
-
 # Flask settings
 CSRF_ENABLED = True
 
@@ -57,3 +58,6 @@ USER_PASSLIB_CRYPTCONTEXT_SCHEMES = ['bcrypt']  # bcrypt for password hashing
 # Flask-User routing settings
 USER_AFTER_LOGIN_ENDPOINT = "main.index"
 USER_AFTER_LOGOUT_ENDPOINT = "main.index"
+
+# Option will be removed once this feature is full implemeted
+ENABLE_CHAR_XML_UPLOAD = False

app/templates/base.html.j2

@@ -90,30 +90,11 @@
       let target_nav = '#{{request.endpoint}}'.replace('\.', '-');
       $(target_nav).addClass('active');
     });
+    // make tooltips with data work
     $(function () {
       $('[data-toggle="tooltip"]').tooltip()
     })
-    {% if config.ALLOW_ANALYTICS %}
-      // Matomo JS analytics
-      var _paq = window._paq = window._paq || [];
-      /* tracker methods like "setCustomDimension" should be called before "trackPageView" */
-      _paq.push(["setDocumentTitle", document.domain + "/" + document.title]);
-      _paq.push(['trackPageView']);
-      _paq.push(['enableLinkTracking']);
-      (function() {
-        var u="https://matomo.aronwk.com/";
-        _paq.push(['setTrackerUrl', u+'matomo.php']);
-        _paq.push(['setSiteId', '3']);
-        var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
-        g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
-      })();
-    {% endif %}
   </script>
-  {% if config.ALLOW_ANALYTICS %}
-    <!-- Matomo no js analytics -->
-    <noscript><p><img src="https://matomo.aronwk.com/matomo.php?idsite=3&amp;rec=1" style="border:0;" alt="" /></p></noscript>
-  {% endif %}
-
 {% endblock %}
 
 </body>

app/templates/character/upload.html.j2

@@ -0,0 +1,22 @@
+{% extends 'base.html.j2' %}
+
+{% block title %}
+  Character XML Upload
+{% endblock title %}
+
+{% block content_before %}
+  Character XML Upload
+{% endblock content_before %}
+
+{% block content %}
+  <h3 style="color: orange;">PROCEED WITH CAUTION</h3>
+  <form method=post>
+    {{ form.csrf_token }}
+    <div class="card shadow-sm mx-auto pb-3 bg-dark border-primary" style="width: 20rem;">
+      <div class="card-body">
+        {{ helper.render_field(form.char_xml) }}
+        {{ helper.render_submit_field(form.submit) }}
+      </div>
+    </div>
+  </form>
+{% endblock  %}

app/templates/header.html.j2

@@ -46,7 +46,7 @@
           <a id='property-index' class='nav-link' href='{{ url_for('properties.index') }}'>Properties</a>
         {% endif %}
 
-        {% if current_user.is_authenticated and current_user.gm_level == 9 and config.REQUIRE_PLAY_KEY %}
+        {% if current_user.is_authenticated and current_user.gm_level >= 5 and config.REQUIRE_PLAY_KEY %}
           {# Play Keys #}
           <a id='play_keys-index' class='nav-link' href='{{ url_for('play_keys.index') }}'>Play Keys</a>
         {% endif %}
@@ -71,8 +71,8 @@
               {% if current_user.is_authenticated and current_user.gm_level >= 8 %}
                 <hr/>
                 <h3 class="text-center">Logs</h3>
-                <a class="dropdown-item text-center" href='{{ url_for('log.activity') }}'>Command Log</a>
-                <a class="dropdown-item text-center" href='{{ url_for('log.command') }}'>Activity Log</a>
+                <a class="dropdown-item text-center" href='{{ url_for('log.command') }}'>Command Log</a>
+                <a class="dropdown-item text-center" href='{{ url_for('log.activity') }}'>Activity Log</a>
                 <a class="dropdown-item text-center" href='{{ url_for('log.audit') }}'>Audit Log</a>
                 <a class="dropdown-item text-center" href='{{ url_for('log.system') }}'>System Log</a>
               {% endif %}

app/templates/ldd/ldd.html.j2

@@ -35,29 +35,6 @@
 
 <script type="text/javascript" src="{{ url_for('static', filename='lddviewer/base64-binary.js') }}"></script>
 
-{% if config.ALLOW_ANALYTICS %}
-  <script>
-    // Matomo JS analytics
-    var _paq = window._paq = window._paq || [];
-    /* tracker methods like "setCustomDimension" should be called before "trackPageView" */
-    _paq.push(["setDocumentTitle", document.domain + "/" + document.title]);
-    _paq.push(['trackPageView']);
-    _paq.push(['enableLinkTracking']);
-    (function() {
-      var u="https://matomo.aronwk.com/";
-      _paq.push(['setTrackerUrl', u+'matomo.php']);
-      _paq.push(['setSiteId', '3']);
-      var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
-      g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
-    })();
-  </script>
-{% endif %}
-
-{% if config.ALLOW_ANALYTICS %}
-  <!-- Matomo no js analytics -->
-  <noscript><p><img src="https://matomo.aronwk.com/matomo.php?idsite=3&amp;rec=1" style="border:0;" alt="" /></p></noscript>
-{% endif %}
-
 <script type='module'>
   import {MTLLoader} from 'https://cdn.jsdelivr.net/npm/three@0.116.0/examples/jsm/loaders/MTLLoader.js'
   import {OBJLoader} from 'https://cdn.jsdelivr.net/npm/three@0.116.0/examples/jsm/loaders/OBJLoader.js'

app/templates/logs/activity.html.j2

@@ -22,7 +22,9 @@
         <tr>
             <th>ID</th>
             <th>Account:Character</th>
-            <th>Command</th>
+            <th>Activity</th>
+            <th>Time</th>
+            <th>Map</th>
         </tr>
     </thead>
     <tbody></tbody>
@@ -38,7 +40,7 @@
             "order": [[0, "desc"]],
             "processing": true,
             "serverSide": true,
-            "ajax": "{{ url_for('log.get_commands') }}",
+            "ajax": "{{ url_for('log.get_activities') }}",
           });
         });
   </script>

app/templates/logs/command.html.j2

@@ -22,9 +22,7 @@
         <tr>
             <th>ID</th>
             <th>Account:Character</th>
-            <th>Activity</th>
-            <th>Time</th>
-            <th>Map</th>
+            <th>Command</th>
         </tr>
     </thead>
     <tbody></tbody>
@@ -40,7 +38,7 @@
             "order": [[0, "desc"]],
             "processing": true,
             "serverSide": true,
-            "ajax": "{{ url_for('log.get_activities') }}",
+            "ajax": "{{ url_for('log.get_commands') }}",
           });
         });
   </script>

app/templates/main/about.html.j2

@@ -4,9 +4,47 @@
 
 {% block content_before %}
   Online Players: {{ online }}
+
 {% endblock %}
 
 {% block content %}
+
+  {# show general zone info to everyone #}
+  {% if zones %}
+    <div class='card mx-auto mt-5 shadow-sm bg-dark border-primary'>
+      <div class="card-body">
+        {% for zone, players in zones.items() %}
+          <div class="row">
+            <div class="col text-right">
+              {{ zone|get_zone_name }}
+            </div>
+            <div class="col">
+              {{ players }}
+            </div>
+          </div>
+        {% endfor %}
+      </div>
+    </div>
+  {% endif %}
+
+  {# only show this info to high level admina #}
+  {% if current_user.gm_level >= 8 and users|length > 0 %}
+    <div class='card mx-auto mt-5 shadow-sm bg-dark border-primary'>
+      <div class="card-body">
+        {% for user in users %}
+          <div class="row">
+            <div class="col text-right">
+              {{ user[0] }}
+            </div>
+            <div class="col">
+              {{ user[1]|get_zone_name }}
+            </div>
+          </div>
+        {% endfor %}
+      </div>
+    </div>
+  {% endif %}
+
   <div class='card mx-auto mt-5 shadow-sm bg-dark border-primary'>
     <div class="card-body">
       <h4 class="text-center">Staff</h4>

app/templates/partials/_account.html.j2

@@ -108,6 +108,12 @@
           </div>
         {% endif %}
       </div>
+    {% elif current_user.gm_level == 9 %}
+      <div class="col">
+        <a role="button" class="btn btn-danger btn btn-block" href='{{ url_for('accounts.pass_reset', id= account_data.id) }}'>
+          Reset User's Password
+        </a>
+      </div>
     {% endif %}
 
     {% if account_data.play_key and current_user.gm_level > 3 and config.REQUIRE_PLAY_KEY %}

app/templates/partials/_character.html.j2

@@ -41,29 +41,35 @@
       </div>
     </div>
     {% if request.endpoint != "characters.view" %}
-    <br/>
-    <div class="row">
-      <div class="col text-center">
-          <a role="button" class="btn btn-primary btn-block"
-            href='{{ url_for('characters.view', id=character.id) }}'>
-            View Character
-          </a>
+      <br/>
+      <div class="row">
+        <div class="col text-center">
+            <a role="button" class="btn btn-primary btn-block"
+              href='{{ url_for('characters.view', id=character.id) }}'>
+              View Character
+            </a>
+        </div>
       </div>
-    </div>
     {% else %}
-    <br/>
-    <a role="button" class="btn btn-primary btn-block"
-      href='{{ url_for('characters.view_xml', id=character.id) }}'>
-      View XML
-    </a>
-    <a role="button" class="btn btn-primary btn-block"
-      href='{{ url_for('characters.get_xml', id=character.id) }}'>
-      Download XML
-    </a>
-    <a role="button" class="btn btn-primary btn-block"
-      href='{{ url_for('accounts.view', id=character.account_id) }}'>
-      View Account: {{character.account.username}}
-    </a>
+      <br/>
+      <a role="button" class="btn btn-primary btn-block"
+        href='{{ url_for('characters.view_xml', id=character.id) }}'>
+        View XML
+      </a>
+      <a role="button" class="btn btn-primary btn-block"
+        href='{{ url_for('characters.get_xml', id=character.id) }}'>
+        Download XML
+      </a>
+      {% if config.ENABLE_CHAR_XML_UPLOAD and current_user.gm_level >= 8 %}
+        <a role="button" class="btn btn-primary btn-block"
+          href='{{ url_for('characters.upload', id=character.id) }}'>
+          Upload XML
+        </a>
+      {% endif %}
+      <a role="button" class="btn btn-primary btn-block"
+        href='{{ url_for('accounts.view', id=character.account_id) }}'>
+        View Account: {{character.account.username}}
+      </a>
     {% endif %}
 
     {% if current_user.gm_level > 2 %}

app/templates/partials/_charxml.html.j2

@@ -63,9 +63,13 @@
         Play time:
       </div>
       <div class="col">
+        {% if character_json.obj.char.attr_time %}
           {{ (character_json.obj.char.attr_time|int/60/60/24)|int }} Days
           {{ (character_json.obj.char.attr_time|int/60/60)|int - ((character_json.obj.char.attr_time|int/60/60/24)|int) * 24}} Hours
           {{ (character_json.obj.char.attr_time|int/60 - (character_json.obj.char.attr_time|int/60/60)|int*60)|int }} Minutes
+        {% else %}
+          None
+        {% endif %}
       </div>
     </div>
     <hr class="bg-primary"/>
@@ -110,9 +114,15 @@
             {# Inv ID 0 - Index: 0 #}
             {% for item in character_json.obj.inv.holdings.in %}
               {% if item.attr_t == "0" %}
-                {% for inv_item in item.i %}
-                  {% include 'partials/charxml/_inv_grid.html.j2' %}
-                {% endfor %}
+                {% if item.i is iterable and (item.i is not string and item.i is not mapping) %}
+                  {% for inv_item in item.i %}
+                    {% include 'partials/charxml/_inv_grid.html.j2' %}
+                  {% endfor %}
+                {% else %}
+                  {% with inv_item=item.i %}
+                    {% include 'partials/charxml/_inv_grid.html.j2' %}
+                  {% endwith %}
+                {% endif %}
               {% endif %}
             {% endfor %}
           </div>
@@ -120,9 +130,15 @@
             {# Inv ID 1 - Index: 1 #}
             {% for item in character_json.obj.inv.holdings.in %}
               {% if item.attr_t == "1" %}
-                {% for inv_item in item.i %}
-                  {% include 'partials/charxml/_inv_grid.html.j2' %}
-                {% endfor %}
+                {% if item.i is iterable and (item.i is not string and item.i is not mapping) %}
+                  {% for inv_item in item.i %}
+                    {% include 'partials/charxml/_inv_grid.html.j2' %}
+                  {% endfor %}
+                {% else %}
+                  {% with inv_item=item.i %}
+                    {% include 'partials/charxml/_inv_grid.html.j2' %}
+                  {% endwith %}
+                {% endif %}
               {% endif %}
             {% endfor %}
           </div>
@@ -130,9 +146,15 @@
             {# Inv ID 14 - Index: 10 #}
             {% for item in character_json.obj.inv.holdings.in %}
               {% if item.attr_t == "14" %}
-                {% for inv_item in item.i %}
-                  {% include 'partials/charxml/_inv_grid.html.j2' %}
-                {% endfor %}
+                {% if item.i is iterable and (item.i is not string and item.i is not mapping) %}
+                  {% for inv_item in item.i %}
+                    {% include 'partials/charxml/_inv_grid.html.j2' %}
+                  {% endfor %}
+                {% else %}
+                  {% with inv_item=item.i %}
+                    {% include 'partials/charxml/_inv_grid.html.j2' %}
+                  {% endwith %}
+                {% endif %}
               {% endif %}
             {% endfor %}
           </div>
@@ -140,9 +162,15 @@
             {# Inv ID 2 - Index: 2 #}
             {% for item in character_json.obj.inv.holdings.in %}
               {% if item.attr_t == "2" %}
-                {% for inv_item in item.i %}
-                  {% include 'partials/charxml/_inv_grid.html.j2' %}
-                {% endfor %}
+                {% if item.i is iterable and (item.i is not string and item.i is not mapping) %}
+                  {% for inv_item in item.i %}
+                    {% include 'partials/charxml/_inv_grid.html.j2' %}
+                  {% endfor %}
+                {% else %}
+                  {% with inv_item=item.i %}
+                    {% include 'partials/charxml/_inv_grid.html.j2' %}
+                  {% endwith %}
+                {% endif %}
               {% endif %}
             {% endfor %}
           </div>
@@ -150,9 +178,15 @@
             {# Inv ID 5 - Index: 6 #}
             {% for item in character_json.obj.inv.holdings.in %}
               {% if item.attr_t == "5" %}
-                {% for inv_item in item.i %}
-                  {% include 'partials/charxml/_inv_grid.html.j2' %}
-                {% endfor %}
+                {% if item.i is iterable and (item.i is not string and item.i is not mapping) %}
+                  {% for inv_item in item.i %}
+                    {% include 'partials/charxml/_inv_grid.html.j2' %}
+                  {% endfor %}
+                {% else %}
+                  {% with inv_item=item.i %}
+                    {% include 'partials/charxml/_inv_grid.html.j2' %}
+                  {% endwith %}
+                {% endif %}
               {% endif %}
             {% endfor %}
           </div>
@@ -160,9 +194,15 @@
             {# Inv ID 7 - Index: 8 #}
             {% for item in character_json.obj.inv.holdings.in %}
               {% if item.attr_t == "7" %}
-                {% for inv_item in item.i %}
-                  {% include 'partials/charxml/_inv_grid.html.j2' %}
-                {% endfor %}
+                {% if item.i is iterable and (item.i is not string and item.i is not mapping) %}
+                  {% for inv_item in item.i %}
+                    {% include 'partials/charxml/_inv_grid.html.j2' %}
+                  {% endfor %}
+                {% else %}
+                  {% with inv_item=item.i %}
+                    {% include 'partials/charxml/_inv_grid.html.j2' %}
+                  {% endwith %}
+                {% endif %}
               {% endif %}
             {% endfor %}
           </div>
@@ -187,10 +227,14 @@
         </button>
       </div>
       <div class="modal-body">
-        {% for zone in character_json.obj.char.zs.s %}
-          {% include 'partials/charxml/_zone_stats.html.j2' %}
-          {{ '<hr class="bg-primary"/>' if not loop.last else "" }}
-        {% endfor %}
+        {% if character_json.obj.char.zs %}
+          {% for zone in character_json.obj.char.zs.s %}
+            {% include 'partials/charxml/_zone_stats.html.j2' %}
+            {{ '<hr class="bg-primary"/>' if not loop.last else "" }}
+          {% endfor %}
+        {% else %}
+          No Stats Yet
+        {% endif %}
       </div>
       <div class="modal-footer">
         <button type="button" class="btn btn-secondary" data-dismiss="modal">Close</button>

app/templates/partials/_gm_level.html.j2

@@ -14,7 +14,7 @@
 {% if gm_level==0 %}
   Player
 {% elif gm_level==1 %}
-  Key Distributor
+  Elevevated Civilan {# Unused #}
 {% elif gm_level==2 %}
   Junior Moderator
 {% elif gm_level==3 %}

app/templates/partials/charxml/_inv_grid.html.j2

@@ -13,7 +13,11 @@
   >
   {% if inv_item.attr_c != "1" %}
     <span class="inventory-count text-bold">
-      {{ inv_item.attr_c }}
+      {%if inv_item.attr_c|int > 999 %}
+        +999
+      {% else %}
+        {{ inv_item.attr_c }}
+      {% endif %}
     </span>
   {% endif %}
   {% if inv_item.attr_b == "true" %}

app/templates/partials/charxml/_item_tooltip.html.j2

@@ -66,3 +66,7 @@
   {% endwith %}
 {% endif %}
 
+{%if inv_item.attr_c|int > 999 %}
+  <br />Count: {{ inv_item.attr_c|numberFormat }}
+{% endif %}
+

entrypoint.bat

@@ -0,0 +1,2 @@
+python3 -m flask db upgrade
+python3 wsgi.py

entrypoint.sh

@@ -1,8 +1,5 @@
 #!/usr/bin/env bash
 
-# unzip brickdb from client to the right places
-unzip -n -q /app/luclient/res/brickdb.zip -d app/luclient/res/
-
 # TODO: preconvert images options
 # TODO: preconvery models options
 

migrations/versions/712d42956a47_initial_migration.py

@@ -77,11 +77,11 @@ def upgrade():
             sa.Column('created_at', mysql.TIMESTAMP(), server_default=sa.text('now()'), nullable=False),
             sa.Column('play_key_id', mysql.INTEGER(), nullable=True),
             sa.Column('mute_expire', mysql.BIGINT(unsigned=True), server_default='0', nullable=False),
-            sa.ForeignKeyConstraint(['play_key_id'], ['play_keys.id'], ondelete='CASCADE'),
             sa.PrimaryKeyConstraint('id'),
             sa.UniqueConstraint('name')
         )
 
+    op.create_foreign_key(None, 'accounts', 'play_keys', ['play_key_id'], ['id'], ondelete='CASCADE')
     op.add_column('accounts', sa.Column('active', sa.BOOLEAN(), server_default='1', nullable=False))
     op.add_column('accounts', sa.Column('email_confirmed_at', sa.DateTime(), nullable=True))
     op.add_column('accounts', sa.Column('email', sa.Unicode(length=255), server_default='', nullable=True))
@@ -103,14 +103,13 @@ def upgrade():
             sa.Column('other_player_id', mysql.TEXT(), nullable=False),
             sa.Column('selection', mysql.TEXT(), nullable=False),
             sa.Column('submitted', mysql.TIMESTAMP(), server_default=sa.text('now()'), nullable=False),
-            sa.ForeignKeyConstraint(['resoleved_by_id'], ['accounts.id'], ondelete='CASCADE'),
             sa.PrimaryKeyConstraint('id')
         )
 
     op.add_column('bug_reports', sa.Column('resolved_time', mysql.TIMESTAMP(), nullable=True))
     op.add_column('bug_reports', sa.Column('resoleved_by_id', sa.Integer(), nullable=True))
     op.add_column('bug_reports', sa.Column('resolution', mysql.TEXT(), nullable=True))
-    op.create_foreign_key(None, 'bug_reports', 'accounts', ['resoleved_by_id'], ['id'])
+    op.create_foreign_key(None, 'bug_reports', 'accounts', ['resoleved_by_id'], ['id'], ondelete='CASCADE')
 
     if 'charinfo' not in tables:
         op.create_table('charinfo',

migrations/versions/a6e42ef03da7_force_play_key_id_to_be_nullable.py

@@ -0,0 +1,33 @@
+"""force play_key_id to be nullable
+
+Revision ID: a6e42ef03da7
+Revises: 8e52b5c7568a
+Create Date: 2022-11-29 19:14:22.645911
+
+"""
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import mysql
+
+# revision identifiers, used by Alembic.
+revision = 'a6e42ef03da7'
+down_revision = '8e52b5c7568a'
+branch_labels = None
+depends_on = None
+
+
+def upgrade():
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.alter_column('accounts', 'play_key_id',
+               existing_type=mysql.INTEGER(display_width=11),
+               nullable=True)
+    # ### end Alembic commands ###
+
+
+def downgrade():
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.alter_column('accounts', 'play_key_id',
+               existing_type=mysql.INTEGER(display_width=11),
+               nullable=False)
+    # ### end Alembic commands ###
+

requirements.txt

@@ -28,7 +28,7 @@ itsdangerous==2.0.1
 Jinja2==3.0.1
 lazy-object-proxy==1.7.1
 libsass==0.21.0
-Mako==1.1.6
+Mako==1.2.2
 MarkupSafe==2.0.1
 marshmallow==3.14.1
 marshmallow-sqlalchemy==0.26.1
@@ -49,7 +49,6 @@ six==1.16.0
 snowballstemmer==2.2.0
 SQLAlchemy==1.4.22
 sqlalchemy-datatables==2.0.1
-SQLAlchemy-Utils==0.38.2
 toml==0.10.2
 tzdata==2021.5
 tzlocal==4.1

wsgi.py

@@ -18,7 +18,7 @@ else:
     from logging.handlers import RotatingFileHandler
     gunicorn_logger = logging.getLogger('gunicorn.error')
     app.logger.handlers = gunicorn_logger.handlers
-    file_handler = RotatingFileHandler('nexus_dashboard.log', maxBytes=1024 * 1024 * 100, backupCount=20)
+    file_handler = RotatingFileHandler('logs/nexus_dashboard.log', maxBytes=1024 * 1024 * 100, backupCount=20)
     formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
     file_handler.setFormatter(formatter)
     app.logger.addHandler(file_handler)