Revert accidental fast-forward merge from v1.49.0rc1

Revert "Sort internal changes in changelog"
Revert "Update CHANGES.md"
Revert "1.49.0rc1"
Revert "Revert "Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505) (#11527)"
Revert "Refactors in `_generate_sync_entry_for_rooms` (#11515)"
Revert "Correctly register shutdown handler for presence workers (#11518)"
Revert "Fix `ModuleApi.looping_background_call` for non-async functions (#11524)"
Revert "Fix 'delete room' admin api to work on incomplete rooms (#11523)"
Revert "Correctly ignore invites from ignored users (#11511)"
Revert "Fix the test breakage introduced by #11435 as a result of concurrent PRs (#11522)"
Revert "Stabilise support for MSC2918 refresh tokens as they have now been merged into the Matrix specification. (#11435)"
Revert "Save the OIDC session ID (sid) with the device on login (#11482)"
Revert "Add admin API to get some information about federation status (#11407)"
Revert "Include bundled aggregations in /sync and related fixes (#11478)"
Revert "Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505)"
Revert "Update backward extremity docs to make it clear that it does not indicate whether we have fetched an events' `prev_events` (#11469)"
Revert "Support configuring the lifetime of non-refreshable access tokens separately to refreshable access tokens. (#11445)"
Revert "Add type hints to `synapse/tests/rest/admin` (#11501)"
Revert "Revert accidental commits to develop."
Revert "Newsfile"
Revert "Give `tests.server.setup_test_homeserver` (nominally!) the same behaviour"
Revert "Move `tests.utils.setup_test_homeserver` to `tests.server`"
Revert "Convert one of the `setup_test_homeserver`s to `make_test_homeserver_synchronous`"
Revert "Disambiguate queries on `state_key` (#11497)"
Revert "Comments on the /sync tentacles (#11494)"
Revert "Clean up tests.storage.test_appservice (#11492)"
Revert "Clean up `tests.storage.test_main` to remove use of legacy code. (#11493)"
Revert "Clean up `tests.test_visibility` to remove legacy code. (#11495)"
Revert "Minor cleanup on recently ported doc pages  (#11466)"
Revert "Add most of the missing type hints to `synapse.federation`. (#11483)"
Revert "Avoid waiting for zombie processes in `synctl stop` (#11490)"
Revert "Fix media repository failing when media store path contains symlinks (#11446)"
Revert "Add type annotations to `tests.storage.test_appservice`. (#11488)"
Revert "`scripts-dev/sign_json`: support for signing events (#11486)"
Revert "Add MSC3030 experimental client and federation API endpoints to get the closest event to a given timestamp (#9445)"
Revert "Port wiki pages to documentation website (#11402)"
Revert "Add a license header and comment. (#11479)"
Revert "Clean-up get_version_string (#11468)"
Revert "Link background update controller docs to summary (#11475)"
Revert "Additional type hints for config module. (#11465)"
Revert "Register the login redirect endpoint for v3. (#11451)"
Revert "Update openid.md"
Revert "Remove mention of OIDC certification from Dex (#11470)"
Revert "Add a note about huge pages to our Postgres doc (#11467)"
Revert "Don't start Synapse master process if `worker_app` is set (#11416)"
Revert "Expose worker & homeserver as entrypoints in `setup.py` (#11449)"
Revert "Bundle relations of relations into the `/relations` result. (#11284)"
Revert "Fix `LruCache` corruption bug with a `size_callback` that can return 0 (#11454)"
Revert "Eliminate a few `Any`s in `LruCache` type hints (#11453)"
Revert "Remove unnecessary `json.dumps` from `tests.rest.admin` (#11461)"
Revert "Merge branch 'master' into develop"

This reverts commit 26b5d23.
This reverts commit bce4220.
This reverts commit 966b5d0.
This reverts commit 088d748.
This reverts commit 14d593f.
This reverts commit 2a3ec6f.
This reverts commit eccc49d.
This reverts commit b1ecd19.
This reverts commit 9c55ded.
This reverts commit 2d42e58.
This reverts commit 2f053f3.
This reverts commit a15a893.
This reverts commit 8b4b153.
This reverts commit 494ebd7.
This reverts commit a77c369.
This reverts commit 4eb7796.
This reverts commit 637df95.
This reverts commit e5f426c.
This reverts commit 8cd68b8.
This reverts commit 6cae125.
This reverts commit 7be88fb.
This reverts commit b3fd99b.
This reverts commit f7ec6e7.
This reverts commit 5640992.
This reverts commit d26808d.
This reverts commit f91624a.
This reverts commit 16d39a5.
This reverts commit 8a4c296.
This reverts commit 49e1356.
This reverts commit d2279f4.
This reverts commit b50e39d.
This reverts commit 858d80b.
This reverts commit 435f044.
This reverts commit f61462e.
This reverts commit a6f1a3a.
This reverts commit 84dc50e.
This reverts commit ed635d3.
This reverts commit 7b62791.
This reverts commit 153194c.
This reverts commit f44d729.
This reverts commit a265fbd.
This reverts commit b9fef1a.
This reverts commit b0eb64f.
This reverts commit f179546.
This reverts commit 70cbb1a.
This reverts commit 42bf020.
This reverts commit 379f265.
This reverts commit 7ff22d6.
This reverts commit 5a0b652.
This reverts commit 432a174.
This reverts commit b14f8a1, reversing
changes made to e713855.

.github/workflows/tests.yml

@@ -374,7 +374,7 @@ jobs:
         working-directory: complement/dockerfiles
 
       # Run Complement
-      - run: go test -v -tags synapse_blacklist,msc2403 ./tests/...
+      - run: go test -v -tags synapse_blacklist,msc2403,msc2946,msc3083 ./tests/...
         env:
           COMPLEMENT_BASE_IMAGE: complement-synapse:latest
         working-directory: complement

CHANGES.md

@@ -1,96 +1,3 @@
-Synapse 1.49.0rc1 (2021-12-07)
-==============================
-
-We've decided to move the existing, somewhat stagnant pages from the GitHub wiki
-to the [documentation website](https://matrix-org.github.io/synapse/latest/).
-
-This was done for two reasons. The first was to ensure that changes are checked by
-multiple authors before being committed (everyone makes mistakes!) and the second
-was visibility of the documentation. Not everyone knows that Synapse has some very
-useful information hidden away in its GitHub wiki pages. Bringing them to the
-documentation website should help with visibility, as well as keep all Synapse documentation
-in one, easily-searchable location.
-
-Note that contributions to the documentation website happen through [GitHub pull
-requests](https://github.com/matrix-org/synapse/pulls). Please visit [#synapse-dev:matrix.org](https://matrix.to/#/#synapse-dev:matrix.org)
-if you need help with the process!
-
-
-Features
---------
-
-- Add [MSC3030](https://github.com/matrix-org/matrix-doc/pull/3030) experimental client and federation API endpoints to get the closest event to a given timestamp. ([\#9445](https://github.com/matrix-org/synapse/issues/9445))
-- Include bundled relation aggregations during a limited `/sync` request and `/relations` request, per [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675). ([\#11284](https://github.com/matrix-org/synapse/issues/11284), [\#11478](https://github.com/matrix-org/synapse/issues/11478))
-- Add plugin support for controlling database background updates. ([\#11306](https://github.com/matrix-org/synapse/issues/11306), [\#11475](https://github.com/matrix-org/synapse/issues/11475), [\#11479](https://github.com/matrix-org/synapse/issues/11479))
-- Support the stable API endpoints for [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946): the room `/hierarchy` endpoint. ([\#11329](https://github.com/matrix-org/synapse/issues/11329))
-- Add admin API to get some information about federation status with remote servers. ([\#11407](https://github.com/matrix-org/synapse/issues/11407))
-- Support expiry of refresh tokens and expiry of the overall session when refresh tokens are in use. ([\#11425](https://github.com/matrix-org/synapse/issues/11425))
-- Stabilise support for [MSC2918](https://github.com/matrix-org/matrix-doc/blob/main/proposals/2918-refreshtokens.md#msc2918-refresh-tokens) refresh tokens as they have now been merged into the Matrix specification. ([\#11435](https://github.com/matrix-org/synapse/issues/11435), [\#11522](https://github.com/matrix-org/synapse/issues/11522))
-- Update [MSC2918 refresh token](https://github.com/matrix-org/matrix-doc/blob/main/proposals/2918-refreshtokens.md#msc2918-refresh-tokens) support to confirm with the latest revision: accept the `refresh_tokens` parameter in the request body rather than in the URL parameters. ([\#11430](https://github.com/matrix-org/synapse/issues/11430))
-- Support configuring the lifetime of non-refreshable access tokens separately to refreshable access tokens. ([\#11445](https://github.com/matrix-org/synapse/issues/11445))
-- Expose `synapse_homeserver` and `synapse_worker` commands as entry points to run Synapse's main process and worker processes, respectively. Contributed by @Ma27. ([\#11449](https://github.com/matrix-org/synapse/issues/11449))
-- `synctl stop` will now wait for Synapse to exit before returning. ([\#11459](https://github.com/matrix-org/synapse/issues/11459), [\#11490](https://github.com/matrix-org/synapse/issues/11490))
-- Extend the "delete room" admin api to work correctly on rooms which have previously been partially deleted. ([\#11523](https://github.com/matrix-org/synapse/issues/11523))
-- Add support for the `/_matrix/client/v3/login/sso/redirect/{idpId}` API from Matrix v1.1. This endpoint was overlooked when support for v3 endpoints was added in Synapse 1.48.0rc1. ([\#11451](https://github.com/matrix-org/synapse/issues/11451))
-
-
-Bugfixes
---------
-
-- Fix using [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) batch sending in combination with event persistence workers. Contributed by @tulir at Beeper. ([\#11220](https://github.com/matrix-org/synapse/issues/11220))
-- Fix a long-standing bug where all requests that read events from the database could get stuck as a result of losing the database connection, properly this time. Also fix a race condition introduced in the previous insufficient fix in Synapse 1.47.0. ([\#11376](https://github.com/matrix-org/synapse/issues/11376))
-- The `/send_join` response now includes the stable `event` field instead of the unstable field from [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083). ([\#11413](https://github.com/matrix-org/synapse/issues/11413))
-- Fix a bug introduced in Synapse 1.47.0 where `send_join` could fail due to an outdated `ijson` version. ([\#11439](https://github.com/matrix-org/synapse/issues/11439), [\#11441](https://github.com/matrix-org/synapse/issues/11441), [\#11460](https://github.com/matrix-org/synapse/issues/11460))
-- Fix a bug introduced in Synapse 1.36.0 which could cause problems fetching event-signing keys from trusted key servers. ([\#11440](https://github.com/matrix-org/synapse/issues/11440))
-- Fix a bug introduced in Synapse 1.47.1 where the media repository would fail to work if the media store path contained any symbolic links. ([\#11446](https://github.com/matrix-org/synapse/issues/11446))
-- Fix an `LruCache` corruption bug, introduced in Synapse 1.38.0, that would cause certain requests to fail until the next Synapse restart. ([\#11454](https://github.com/matrix-org/synapse/issues/11454))
-- Fix a long-standing bug where invites from ignored users were included in incremental syncs. ([\#11511](https://github.com/matrix-org/synapse/issues/11511))
-- Fix a regression in Synapse 1.48.0 where presence workers would not clear their presence updates over replication on shutdown. ([\#11518](https://github.com/matrix-org/synapse/issues/11518))
-- Fix a regression in Synapse 1.48.0 where the module API's `looping_background_call` method would spam errors to the logs when given a non-async function. ([\#11524](https://github.com/matrix-org/synapse/issues/11524))
-
-
-Updates to the Docker image
----------------------------
-
-- Update `Dockerfile-workers` to healthcheck all workers in the container. ([\#11429](https://github.com/matrix-org/synapse/issues/11429))
-
-
-Improved Documentation
-----------------------
-
-- Update the media repository documentation. ([\#11415](https://github.com/matrix-org/synapse/issues/11415))
-- Update section about backward extremities in the room DAG concepts doc to correct the misconception about backward extremities indicating whether we have fetched an events' `prev_events`. ([\#11469](https://github.com/matrix-org/synapse/issues/11469))
-
-
-Internal Changes
-----------------
-
-- Add `Final` annotation to string constants in `synapse.api.constants` so that they get typed as `Literal`s. ([\#11356](https://github.com/matrix-org/synapse/issues/11356))
-- Add a check to ensure that users cannot start the Synapse master process when `worker_app` is set. ([\#11416](https://github.com/matrix-org/synapse/issues/11416))
-- Add a note about postgres memory management and hugepages to postgres doc. ([\#11467](https://github.com/matrix-org/synapse/issues/11467))
-- Add missing type hints to `synapse.config` module. ([\#11465](https://github.com/matrix-org/synapse/issues/11465))
-- Add missing type hints to `synapse.federation`. ([\#11483](https://github.com/matrix-org/synapse/issues/11483))
-- Add type annotations to `tests.storage.test_appservice`. ([\#11488](https://github.com/matrix-org/synapse/issues/11488), [\#11492](https://github.com/matrix-org/synapse/issues/11492))
-- Add type annotations to some of the configuration surrounding refresh tokens. ([\#11428](https://github.com/matrix-org/synapse/issues/11428))
-- Add type hints to `synapse/tests/rest/admin`. ([\#11501](https://github.com/matrix-org/synapse/issues/11501))
-- Add type hints to storage classes. ([\#11411](https://github.com/matrix-org/synapse/issues/11411))
-- Add wiki pages to documentation website. ([\#11402](https://github.com/matrix-org/synapse/issues/11402))
-- Clean up `tests.storage.test_main` to remove use of legacy code. ([\#11493](https://github.com/matrix-org/synapse/issues/11493))
-- Clean up `tests.test_visibility` to remove legacy code. ([\#11495](https://github.com/matrix-org/synapse/issues/11495))
-- Convert status codes to `HTTPStatus` in `synapse.rest.admin`. ([\#11452](https://github.com/matrix-org/synapse/issues/11452), [\#11455](https://github.com/matrix-org/synapse/issues/11455))
-- Extend the `scripts-dev/sign_json` script to support signing events. ([\#11486](https://github.com/matrix-org/synapse/issues/11486))
-- Improve internal types in push code. ([\#11409](https://github.com/matrix-org/synapse/issues/11409))
-- Improve type annotations in `synapse.module_api`. ([\#11029](https://github.com/matrix-org/synapse/issues/11029))
-- Improve type hints for `LruCache`. ([\#11453](https://github.com/matrix-org/synapse/issues/11453))
-- Preparation for database schema simplifications: disambiguate queries on `state_key`. ([\#11497](https://github.com/matrix-org/synapse/issues/11497))
-- Refactor `backfilled` into specific behavior function arguments (`_persist_events_and_state_updates` and downstream calls). ([\#11417](https://github.com/matrix-org/synapse/issues/11417))
-- Refactor `get_version_string` to fix-up types and duplicated code. ([\#11468](https://github.com/matrix-org/synapse/issues/11468))
-- Refactor various parts of the `/sync` handler. ([\#11494](https://github.com/matrix-org/synapse/issues/11494), [\#11515](https://github.com/matrix-org/synapse/issues/11515))
-- Remove unnecessary `json.dumps` from `tests.rest.admin`. ([\#11461](https://github.com/matrix-org/synapse/issues/11461))
-- Save the OpenID Connect session ID on login. ([\#11482](https://github.com/matrix-org/synapse/issues/11482))
-- Update and clean up recently ported documentation pages. ([\#11466](https://github.com/matrix-org/synapse/issues/11466))
-
-
 Synapse 1.48.0 (2021-11-30)
 ===========================
 

debian/changelog

@@ -1,9 +1,3 @@
-matrix-synapse-py3 (1.49.0~rc1) stable; urgency=medium
-
-  * New synapse release 1.49.0~rc1.
-
- -- Synapse Packaging team <packages@matrix.org>  Tue, 07 Dec 2021 13:52:21 +0000
-
 matrix-synapse-py3 (1.48.0) stable; urgency=medium
 
   * New synapse release 1.48.0.

docker/Dockerfile-workers

@@ -21,6 +21,3 @@ VOLUME ["/data"]
 # files to run the desired worker configuration. Will start supervisord.
 COPY ./docker/configure_workers_and_start.py /configure_workers_and_start.py
 ENTRYPOINT ["/configure_workers_and_start.py"]
-
-HEALTHCHECK --start-period=5s --interval=15s --timeout=5s \
-    CMD /bin/sh /healthcheck.sh

docker/configure_workers_and_start.py

@@ -474,16 +474,10 @@ def generate_worker_files(environ, config_path: str, data_dir: str):
 
     # Determine the load-balancing upstreams to configure
     nginx_upstream_config = ""
-
-    # At the same time, prepare a list of internal endpoints to healthcheck
-    # starting with the main process which exists even if no workers do.
-    healthcheck_urls = ["http://localhost:8080/health"]
-
     for upstream_worker_type, upstream_worker_ports in nginx_upstreams.items():
         body = ""
         for port in upstream_worker_ports:
             body += "    server localhost:%d;\n" % (port,)
-            healthcheck_urls.append("http://localhost:%d/health" % (port,))
 
         # Add to the list of configured upstreams
         nginx_upstream_config += NGINX_UPSTREAM_CONFIG_BLOCK.format(
@@ -516,13 +510,6 @@ def generate_worker_files(environ, config_path: str, data_dir: str):
         worker_config=supervisord_config,
     )
 
-    # healthcheck config
-    convert(
-        "/conf/healthcheck.sh.j2",
-        "/healthcheck.sh",
-        healthcheck_urls=healthcheck_urls,
-    )
-
     # Ensure the logging directory exists
     log_dir = data_dir + "/logs"
     if not os.path.exists(log_dir):

docs/SUMMARY.md

@@ -44,7 +44,6 @@
         - [Presence router callbacks](modules/presence_router_callbacks.md)
         - [Account validity callbacks](modules/account_validity_callbacks.md)
         - [Password auth provider callbacks](modules/password_auth_provider_callbacks.md)
-        - [Background update controller callbacks](modules/background_update_controller_callbacks.md)
         - [Porting a legacy module to the new interface](modules/porting_legacy_module.md)
     - [Workers](workers.md)
       - [Using `synctl` with Workers](synctl_workers.md)
@@ -65,15 +64,9 @@
       - [Statistics](admin_api/statistics.md)
       - [Users](admin_api/user_admin_api.md)
       - [Server Version](admin_api/version_api.md)
-      - [Federation](usage/administration/admin_api/federation.md)
     - [Manhole](manhole.md)
     - [Monitoring](metrics-howto.md)
-    - [Understanding Synapse Through Grafana Graphs](usage/administration/understanding_synapse_through_grafana_graphs.md)
-    - [Useful SQL for Admins](usage/administration/useful_sql_for_admins.md)
-    - [Database Maintenance Tools](usage/administration/database_maintenance_tools.md)
-    - [State Groups](usage/administration/state_groups.md) 
     - [Request log format](usage/administration/request_log.md)
-    - [Admin FAQ](usage/administration/admin_faq.md)
     - [Scripts]()
 
 # Development
@@ -101,4 +94,3 @@
 
 # Other
   - [Dependency Deprecation Policy](deprecation_policy.md)
-  - [Running Synapse on a Single-Board Computer](other/running_synapse_on_single_board_computers.md)

docs/development/room-dag-concepts.md

@@ -38,15 +38,16 @@ Most-recent-in-time events in the DAG which are not referenced by any other even
 The forward extremities of a room are used as the `prev_events` when the next event is sent.
 
 
-## Backward extremity
+## Backwards extremity
 
 The current marker of where we have backfilled up to and will generally be the
-`prev_events` of the oldest-in-time events we have in the DAG. This gives a starting point when
-backfilling history.
+oldest-in-time events we know of in the DAG.
 
-When we persist a non-outlier event, we clear it as a backward extremity and set
-all of its `prev_events` as the new backward extremities if they aren't already
-persisted in the `events` table.
+This is an event where we haven't fetched all of the `prev_events` for.
+
+Once we have fetched all of its `prev_events`, it's unmarked as a backwards
+extremity (although we may have formed new backwards extremities from the prev
+events during the backfilling process).
 
 
 ## Outliers
@@ -55,7 +56,8 @@ We mark an event as an `outlier` when we haven't figured out the state for the
 room at that point in the DAG yet.
 
 We won't *necessarily* have the `prev_events` of an `outlier` in the database,
-but it's entirely possible that we *might*.
+but it's entirely possible that we *might*. The status of whether we have all of
+the `prev_events` is marked as a [backwards extremity](#backwards-extremity).
 
 For example, when we fetch the event auth chain or state for a given event, we
 mark all of those claimed auth events as outliers because we haven't done the

docs/media_repository.md

@@ -2,80 +2,29 @@
 
 *Synapse implementation-specific details for the media repository*
 
-The media repository
- * stores avatars, attachments and their thumbnails for media uploaded by local
-   users.
- * caches avatars, attachments and their thumbnails for media uploaded by remote
-   users.
- * caches resources and thumbnails used for
-   [URL previews](development/url_previews.md).
+The media repository is where attachments and avatar photos are stored.
+It stores attachment content and thumbnails for media uploaded by local users.
+It caches attachment content and thumbnails for media uploaded by remote users.
 
-All media in Matrix can be identified by a unique
-[MXC URI](https://spec.matrix.org/latest/client-server-api/#matrix-content-mxc-uris),
-consisting of a server name and media ID:
-```
-mxc://<server-name>/<media-id>
-```
+## Storage
 
-## Local Media
-Synapse generates 24 character media IDs for content uploaded by local users.
-These media IDs consist of upper and lowercase letters and are case-sensitive.
-Other homeserver implementations may generate media IDs differently.
+Each item of media is assigned a `media_id` when it is uploaded.
+The `media_id` is a randomly chosen, URL safe 24 character string.
 
-Local media is recorded in the `local_media_repository` table, which includes
-metadata such as MIME types, upload times and file sizes.
-Note that this table is shared by the URL cache, which has a different media ID
-scheme.
+Metadata such as the MIME type, upload time and length are stored in the
+sqlite3 database indexed by `media_id`.
 
-### Paths
-A file with media ID `aabbcccccccccccccccccccc` and its `128x96` `image/jpeg`
-thumbnail, created by scaling, would be stored at:
-```
-local_content/aa/bb/cccccccccccccccccccc
-local_thumbnails/aa/bb/cccccccccccccccccccc/128-96-image-jpeg-scale
-```
+Content is stored on the filesystem under a `"local_content"` directory.
 
-## Remote Media
-When media from a remote homeserver is requested from Synapse, it is assigned
-a local `filesystem_id`, with the same format as locally-generated media IDs,
-as described above.
+Thumbnails are stored under a `"local_thumbnails"` directory.
 
-A record of remote media is stored in the `remote_media_cache` table, which
-can be used to map remote MXC URIs (server names and media IDs) to local
-`filesystem_id`s.
+The item with `media_id` `"aabbccccccccdddddddddddd"` is stored under
+`"local_content/aa/bb/ccccccccdddddddddddd"`. Its thumbnail with width
+`128` and height `96` and type `"image/jpeg"` is stored under
+`"local_thumbnails/aa/bb/ccccccccdddddddddddd/128-96-image-jpeg"`
 
-### Paths
-A file from `matrix.org` with `filesystem_id` `aabbcccccccccccccccccccc` and its
-`128x96` `image/jpeg` thumbnail, created by scaling, would be stored at:
-```
-remote_content/matrix.org/aa/bb/cccccccccccccccccccc
-remote_thumbnail/matrix.org/aa/bb/cccccccccccccccccccc/128-96-image-jpeg-scale
-```
-Older thumbnails may omit the thumbnailing method:
-```
-remote_thumbnail/matrix.org/aa/bb/cccccccccccccccccccc/128-96-image-jpeg
-```
-
-Note that `remote_thumbnail/` does not have an `s`.
-
-## URL Previews
-See [URL Previews](development/url_previews.md) for documentation on the URL preview
-process.
-
-When generating previews for URLs, Synapse may download and cache various
-resources, including images. These resources are assigned temporary media IDs
-of the form `yyyy-mm-dd_aaaaaaaaaaaaaaaa`, where `yyyy-mm-dd` is the current
-date and `aaaaaaaaaaaaaaaa` is a random sequence of 16 case-sensitive letters.
-
-The metadata for these cached resources is stored in the
-`local_media_repository` and `local_media_repository_url_cache` tables.
-
-Resources for URL previews are deleted after a few days.
-
-### Paths
-The file with media ID `yyyy-mm-dd_aaaaaaaaaaaaaaaa` and its `128x96`
-`image/jpeg` thumbnail, created by scaling, would be stored at:
-```
-url_cache/yyyy-mm-dd/aaaaaaaaaaaaaaaa
-url_cache_thumbnails/yyyy-mm-dd/aaaaaaaaaaaaaaaa/128-96-image-jpeg-scale
-```
+Remote content is cached under `"remote_content"` directory. Each item of
+remote content is assigned a local `"filesystem_id"` to ensure that the
+directory structure `"remote_content/server_name/aa/bb/ccccccccdddddddddddd"`
+is appropriate. Thumbnails for remote content are stored under
+`"remote_thumbnail/server_name/..."`