notes

------ RESUMING DEVELOPMENT AFTER A PAUSE -----------

no need to run drush serve or anything. so long as the containers are running, you can access the site at http://127.0.0.1:8080
verify containers are running: docker ps
now jump down to the redeployment section

-----------------------------------------------------


DOCKER/COMPOSER COMMANDS:

docker-compose stop      # Pauses containers (they still exist), use that end of day if necessary
docker-compose start     # Resumes them
docker-compose restart

docker-compose down           # Stops and removes containers, stops the network, database and data persist
docker-compose down -v        # Also removes volumes, use for a fresh start
docker-compose -f docker-compose.yaml -f docker-compose.dev.yaml up 		# use this in dev. composer specs are split because we don't want to use too many bind mounts in prod
docker compose -f docker-compose.yaml up -d									# use this in prod
note that on prod there's a third container for the Caddy
docker-compose up --build -d  # Starts or resumes your containers and builds Dockerfile
docker-compose up -d	      # Starts or resumes your containers
docker-compose build --no-cache drupal 									# eventually you'll want to rebuild the container locally. need to ignore cache for that, so up -d will fail

docker ps				list running containers (and get their names)
docker cp container_id:path local_path		copy contents to/from container
docker cp b1a5a7c9b1b8:/opt/drupal/web/sites/default/files/config_gCohTr204_tUOHEoCSoqQdwbSTu87xtRCQZHJalG7RX3qSVKy3vy5Se3bI1B9pRplYijH-tFCA/sync backup/

docker exec -it drupal-platform_drupal_1  bash		enter the container
docker exec -it drupal-platform_db_1  bash		enter the database container?


INSIDE THE CONTAINER:

docker exec or composer		Use commands inside the running container, update code or modules

#must run these from project root (on container)! that's where composer.json is found.
composer require drupal/group
composer require drupal/bootstrap5


PRE-DEPLOY:

# test build
docker-compose down
docker-compose build --no-cache drupal
composer require drush/drush
. post_install_steps.sh
drush cr

# build a lockfile from package versions specified in composer.json (run locally!)
	composer update/install --no-interaction (update may change versions, so prefer install)
	gitignore vendor/ and web/
	commit .gitattributes, .editorconfig, and composer.lock
to do this you will need to install composer on local machine:
	sudo apt update
	sudo apt install -y php-cli unzip curl
	curl -sS https://getcomposer.org/installer -o composer-setup.php
	sudo php composer-setup.php --install-dir=/usr/local/bin --filename=composer
	rm composer-setup.php
	composer --version
if you get an error about missing PHP extensions, install with e.g.:
	sudo apt update
	sudo apt install -y php8.3-xml php8.3-curl
	sudo apt install -y php8.3-zip php8.3-mbstring php8.3-gd php8.3-intl php8.3-mysql


BACKUPS:

# (Use maintenance mode when upgrading, importing or exporting content, modifying a theme, modifying content types, and making backups.)
# eventually you'll want to rebuild the container locally. need to ignore cache for that, so up -d will fail
# need this: apt-get update && apt-get install -y --no-install-recommends mariadb-client
#	but don't need it for production
# can put secrets in settings.local.php, but whatevs
# for regular-basis backups during development, sql:dump is probs fine
# following code backs up files and database, excluding the settings file which contains secrets.

drush sset system.maintenance_mode 1
php -d memory_limit=-1 /opt/drupal/vendor/drush/drush/drush.php archive:dump --destination=/var/www/html/modules/custom/backup.tar.gz --exclude-code-paths=web/sites/default/settings.php
drush sset system.maintenance_mode 0
E.G. (increment): mv modules/custom/backup.tar.gz backup2.tar.gz
docker cp this out periodically?


CONTRIB UPDATES (backup first):

when GUI updating fails:
drush sset system.maintenance_mode 1
e.g.:
	composer update drupal/bootstrap5 --with-all-dependencies
OR (esp. if that fails [due to fixed version?]):
	composer require drupal/bootstrap5:^5.3 --with-all-dependencies
drush updb -y
might need: drush cron
might need: drush ev 'Drupal::service("update.manager")->refreshUpdateData();'
drush cr
drush sset system.maintenance_mode 0


CORE UPDATES (clear all errors and backup first):

drush sset system.maintenance_mode 1
TRY:
	composer update "drupal/core-*" --with-all-dependencies
IF IT FAILS, it's probaby because you've fixed the version? so use require: 
composer require \
  drupal/core-recommended:^10.5 \
  drupal/core-composer-scaffold:^10.5 \
  drupal/core-project-message:^10.5 \
  --with-all-dependencies
drush updb -y
drush cron
drush ev 'Drupal::service("update.manager")->refreshUpdateData();'
drush cr
drush sset system.maintenance_mode 0


OTHER

drush en <module>				enable module (-y flag if desired)
drush pm:uninstall <module>		always run this before removing module files!
drush wd-show					logs
drush pm:list --status=enabled --type=module


REDEPLOY --------------------------------

first, sync remote changes made through the live admin interface:

1. open tmux; remote into the prod server (swislrserv), open tmux within? just want to be able to copy out easily
2. exec into the prod container
3. drush cex
4. docker cp to the repo.
	make sure repo is clean, then rm -r existing backup/sync/ (full path below)
	from host: docker cp f49b478c40e9:/opt/drupal/config/sync backup/
	dont forget about .htaccess file
	it might have nested sync/ dirs. the outer one has a derelict smtp.settings.php, so we want inner contents
5. push/pull down to local
6. docker cp to the container 
	same deal (seriously, rmrf is the only way to ensure no derelict files)
	docker cp backup/sync/ b1a5a7c9b1b8:/opt/drupal/web/sites/default/files/config_gCohTr204_tUOHEoCSoqQdwbSTu87xtRCQZHJalG7RX3qSVKy3vy5Se3bI1B9pRplYijH-tFCA/
7. drush config:status (to see what's about to change before you import)
8. locally, drush cim from within the test container

second, updates:
	never do these on prod! they'll get wiped when you push (unless you perform the above steps, but safer this way)
	admin interface keeps failing. use code. follow the BACKUPS, CONTRIB_UPDATES, and CORE_UPDATES sections above before proceeding

after development:

inside dev container:
	drush cex
	drush cim --diff 											(verify no configs out of sync)
	drush pm:list --type=module --status=enabled | grep swislr 	(verify custom modules are enabled and error-free)
	drush cr
	drush watchdog:show --severity=3 --count=20             	(check for PHP errors)
	composer update 											(rewrite lockfile based on composer.json)
	composer validate --no-check-all                            (verify .lock is current)

dev host:
	docker cp  b87013a961c9:/opt/drupal/composer.json .
	docker cp  b87013a961c9:/opt/drupal/composer.lock .
	commit, push

prod container:
	to get into container, run this on host:
		docker exec -it swislr_seek-drupal-1 bash
	drush sset system.maintenance_mode 1 -y && drush cr

prod host:
	OLD METHOD: docker compose exec db mariadb-dump --skip-ssl -u swislr -p swislr > pre-deploy-backup.sql
		INSTEAD DO THIS: bash backup/backup_site.sh
	git pull (no need to docker cp, because composer.json and lock are copied in Dockerfile)
	docker compose pull (necessary? risky?)
		if building fresh (--no-cache), probably chill, but could introduce php/apache changes. skipping for now.
	docker compose build --no-cache drupal
		caching is fine for routine future deploys, but so much has changed this time that it's best to build fresh
	docker compose up -d

prod container: (with modifications, all of this could be executed from the host)
	drush updb -y (always run. if no updates, is a no-op)
	verify:
		drush sqlq "SELECT DISTINCT collection FROM config" --extra="--skip-ssl" (this is just a test to confirm we don't have a rogue sync/ collection in the db again. should be an empty string. if error, read on)
			in container: drush sqlq "DELETE FROM config WHERE collection = 'sync'" --extra="--skip-ssl"
	drush cr
	at this point, still have to step through some of the "deploy" steps below. i think just:
		. post_install_steps.sh (should be quite automatable now. only needs the drush en lines when there's a new module)
	drush cim
	drush cr
	drush config:status 								(check configs are synced and clean)
	drush watchdog:show --severity=3 --count=20         (check for PHP errors)
	drush cron
	drush sset system.maintenance_mode 0 -y && drush cr

if there are any new navbar items, still need to add them manually. make sure forum still appears!
check status report
make sure basic site config is correct [config -> system -> basic site settings]
	email should be: mail@swislr.org
	what about: change "adhoc emails" in config -> workflow -> cont mod notify: emily.bernhardt@duke.edu,spencer.rhea@duke.edu,odonnellk25@ecu.edu
sftp backups to local

-----------------------------------------------------------------------------------------------

DEPLOY (* steps are now automated, ~ steps are only necessary for initial server setup, ! steps probably aren't necessary at all, ? = unknown)

~	start with hax/servers/server_setup.txt
~	then clone repo. 
!	recreate .env

might have to change docker-compose.yaml to accept traffic on port 80
?	sudo ufw enable
?	sudo ufw allow 80/tcp
?	sudo ufw allow 443/tcp
?	sudo ufw allow ssh
?	sudo ufw status

#don't install default docker. update the next block based on this site: https://docs.docker.com/engine/install/ubuntu/
~	sudo apt-get update && sudo apt-get install -y docker.io docker-compose
~	sudo usermod -aG docker $USER #(log out and back in)
~	docker-compose up -d db
~	docker-compose -f docker-compose.yaml build --no-cache drupal
~	docker-compose up -d drupal
~	docker-compose up -d caddy

*	docker exec -it swislr_seek_drupal_1  bash #(e.g.; might have to use container ID)
*	composer require drush/drush #(supposedly shouldn't need this, becuase drush is already included)

?	#if you have to change anything in the docker-compose.yaml, you'll probably have to wade through all this insane shit:
?	#docker-compose stop drupal
?	#docker-compose rm -f drupal
?	#docker-compose up -d --force-recreate --remove-orphans drupal
?	#(this will fail)
?	#docker images | awk '/drupal|swislr/ {print $3}' | xargs -r docker rmi
?	#(this will totally fail)
?	#docker-compose up -d --build drupal
?	#(but then this will still work)

*	visit seek.swislr.org for database setup. copy names/pws from .env.
*	database host is the name of the database service as specified in docker-compose.yaml. in this case "db"

*	site name: SWISLR Seek
*	site email: mail@swislr.org
*	admin name: swislradmin

*	add to web/sites/default/settings.php:
*	$settings['trusted_host_patterns'] = [
*	  '^seek\.swislr\.org$',
*	  '^www.seek.swislr.org$',
*	  '^104.236.33.140$',
*	];
*	$settings['file_private_path'] = '/var/www/private';


*	#might also need to point it to the correct config directory in web/sites/default/settings.php
*	$settings['config_sync_directory'] = 'sites/default/files/config_kpbYqFzrMb1MfEyjwlhWR5qsQwkPR2JWrlBFlnUxEOk9LO7OvI0qsWLbzDKO2NwBAwhMbCNb3w/sync';


. post_install_steps.sh

*	copy over config files, e.g.:
*	docker cp backup/sync/ b1a5a7c9b1b8:/opt/drupal/web/sites/default/files/config_<hash>/

*	now, in sync/ on the container:
*	grep '^uuid:' system.site.yml #copy the resulting uuid and use it in the next step
*	drush cset system.site uuid "<that uuid>" -y

?	add to pathauto.pattern.story.yml: (though this will break local version), because that entity doesn't exist locally
?	id: story
?	label: 'Story URL pattern'
?	type: 'canonical_entities:node'

after this is is finally possible to run drush cim

check status report
make sure basic site config is correct (e.g. email address for the site)
gotta manually build navbar links? why?

ideal ufw setup:
To                         Action      From
--                         ------      ----
80/tcp                     ALLOW       Anywhere                  
22/tcp                     ALLOW       Anywhere                  
443/tcp                    ALLOW       Anywhere                  
80/tcp (v6)                ALLOW       Anywhere (v6)             
22/tcp (v6)                ALLOW       Anywhere (v6)             
443/tcp (v6)               ALLOW       Anywhere (v6)             


also make sure the container contains the right composer.json (and composer.lock if that already exists in the repo)
	the container seems to get built with defaults for these, and i'm not sure if i've written the copy step into the build yet


---------------- STEPS THAT MIGHT BE DANGLING. REVISIT THIS STUFF -----------------

add to /var/www/html/sites/default/settings.php:
$settings['trusted_host_patterns'] = [
  '^localhost$',
  '^127\.0\.0\.1$',
  '^swislr\.org$',
  '^www\.swislr\.org$',
];

#make a place for private files (now configured in docker-compose.yaml, but still need to add line to settings.php)
cd /opt
mkdir drupal_private
chown www-data:www-data drupal_private/
chmod 700 drupal_private
#add this to web/sites/default/settings.php:
$settings['file_private_path'] = '/opt/drupal_private';
drush cr

#enable custom modules
mkdir -p /var/www/html/modules/custom

might need to change ownership for all the files that are bind mounted

when you develop themes or modules, do so outside the container, then mount from within
volumes:
  - ./themes:/var/www/html/themes
  - ./modules:/var/www/html/modules

is this contradictory? To develop themes or modules, mount them directly from your host machine instead of named volumes.
For production, consider replacing `volumes:` with bind mounts and add HTTPS, caching (e.g., Varnish), and backup mechanisms.

in dockerfile, build custom modules/themes (or anything that will change frequently) last.
that way, the system can cache the earlier steps and build will be faster


---- DIRECTORY ------------
drupal home for my first build was /opt/drupal/web (and i think the dir "drupal" was specified by me?)
alongside this home ("web") folder are vendor/, composer.json, and composer.lock

1. core/
    Contains built-in themes, modules, routing, and services
    Leave this untouched; update it via Composer when needed
2. modules/
	modules/contrib/ → For downloaded modules (e.g., Group, Webform)
	modules/custom/ → For modules you write yourself
    These folders don’t exist by default, but you can create them.
3. themes/
	themes/contrib/ → Downloaded themes
	themes/custom/ → Custom themes you create
	Start with something like Classy, Bootstrap 5, or Olivero
		Extend it as a subtheme
	find themes here: https://www.drupal.org/project/project_theme
	install like so: composer require drupal/bootstrap5
4. sites/default/
        settings.php: site config (DB, trusted hosts, etc.)
        services.yml: service overrides (e.g., Twig debugging)
        files/: uploaded user content
    You can have multi-site setups, each under sites/example.com/, but default is typical
5. profiles/
    Installation profiles (like pre-built site templates)
    Ignore this unless you're distributing your own "starter kit"




-------- modules

find modules here
	https://www.drupal.org/project/project_module

install like so
	composer require drupal/group


Group	Create collaborative spaces (research groups, labs, teams) with their own content and permissions
File Entity	Manage uploaded files as standalone entities (with metadata, revisions)
IMCE	User-friendly file browser/uploader for images/docs
Media	Standardized media management (images, files, videos)
Private Files	Restrict access to uploaded files
Content Moderation	Workflow for draft/review/publish
Webform	Flexible surveys, data collection, and contact forms
Discussion	Enable commenting and threads on content
Search API + Solr	Powerful search for content and files
CAS / SAML / LDAP	Institutional login integration (if needed)