Saner TOTP provisioning

A user's e-mail address may be fluid, and the site "instance name"
may be strange or change regularly. There's no reason to use these
over the user's stable ID and the site's stable hostname for TOTP
parameters. Even if the system is built to TOLERATE changes (as it
is -- I tested it), it seems much more elegant to have these para-
meters as stable identifiers.

CHANGELOG.md

@@ -14,7 +14,27 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ### Removed
 
-## 2.4.0 - 2021-08-xx
+## 2.4.1 - 2021-08-29
+
+### Changed
+- Make `mix pleroma.database set_text_search_config` run concurrently and indefinitely
+
+### Added
+- AdminAPI: Missing configuration description for StealEmojiPolicy
+
+### Fixed
+- MastodonAPI: Stream out Create activities
+- MRF ObjectAgePolicy: Fix pattern matching on "published"
+- TwitterAPI: Make `change_password` and `change_email` require params on body instead of query
+- Subscription(Bell) Notifications: Don't create from Pipeline Ingested replies
+- AdminAPI: Fix rendering reports containing a `nil` object
+- Mastodon API: Activity Search fallbacks on status fetching after a DB Timeout/Error
+- Mastodon API: Fix crash in Streamer related to reblogging
+- AdminAPI: List available frontends when `static/frontends` folder is missing
+- Make activity search properly use language-aware GIN indexes
+- AdminAPI: Fix suggestions for MRF Policies
+
+## 2.4.0 - 2021-08-08
 
 ### Changed
 

docs/configuration/howto_search_cjk.md

@@ -5,7 +5,7 @@ Pleroma's full text search feature is powered by PostgreSQL's native [text searc
 
 ## Setup and test the new search config
 
-In most cases, you would need an extension installed to support parsing CJK text. Here are a few extension you may choose from, or you are more than welcome to share additional ones you found working for you with the rest of Pleroma community.
+In most cases, you would need an extension installed to support parsing CJK text. Here are a few extensions you may choose from, or you are more than welcome to share additional ones you found working for you with the rest of Pleroma community.
 
  * [a generic n-gram parser](https://github.com/huangjimmy/pg_cjk_parser) supports Simplifed/Traditional Chinese, Japanese, and Korean
  * [a Korean parser](https://github.com/i0seph/textsearch_ko) based on mecab
@@ -34,7 +34,7 @@ Check output of the query, and see if it matches your expectation.
     mix pleroma.database set_text_search_config YOUR.CONFIG
     ```
 
-Note: index update may take a while.
+Note: index update may take a while, and it can be done while the instance is up and running, so you may restart db connection as soon as you see `Recreate index` in task output.
 
 ## Restart database connection
 Since some changes above will only apply with a new database connection, you will have to restart either Pleroma or PostgreSQL process, or use `pg_terminate_backend` SQL command without restarting either. 

docs/installation/alpine_linux_en.md

@@ -1,4 +1,7 @@
 # Installing on Alpine Linux
+
+{! backend/installation/otp_vs_from_source_source.include !}
+
 ## Installation
 
 This guide is a step-by-step installation guide for Alpine Linux. The instructions were verified against Alpine v3.10 standard image. You might miss additional dependencies if you use `netboot` instead.

docs/installation/arch_linux_en.md

@@ -1,4 +1,7 @@
 # Installing on Arch Linux
+
+{! backend/installation/otp_vs_from_source_source.include !}
+
 ## Installation
 
 This guide will assume that you have administrative rights, either as root or a user with [sudo permissions](https://wiki.archlinux.org/index.php/Sudo). If you want to run this guide with root, ignore the `sudo` at the beginning of the lines, unless it calls a user like `sudo -Hu pleroma`; in this case, use `su <username> -s $SHELL -c 'command'` instead.

docs/installation/debian_based_en.md

@@ -1,4 +1,7 @@
 # Installing on Debian Based Distributions
+
+{! backend/installation/otp_vs_from_source_source.include !}
+
 ## Installation
 
 This guide will assume you are on Debian 11 (“bullseye”) or later. This guide should also work with Ubuntu 18.04 (“Bionic Beaver”) and later. It also assumes that you have administrative rights, either as root or a user with [sudo permissions](https://www.digitalocean.com/community/tutorials/how-to-add-delete-and-grant-sudo-privileges-to-users-on-a-debian-vps). If you want to run this guide with root, ignore the `sudo` at the beginning of the lines, unless it calls a user like `sudo -Hu pleroma`; in this case, use `su <username> -s $SHELL -c 'command'` instead.

docs/installation/gentoo_en.md

@@ -1,4 +1,7 @@
 # Installing on Gentoo GNU/Linux
+
+{! backend/installation/otp_vs_from_source_source.include !}
+
 ## Installation
 
 This guide will assume that you have administrative rights, either as root or a user with [sudo permissions](https://wiki.gentoo.org/wiki/Sudo). Lines that begin with `#` indicate that they should be run as the superuser. Lines using `$` should be run as the indicated user, e.g. `pleroma$` should be run as the `pleroma` user.

docs/installation/migrating_from_source_otp_en.md

@@ -1,7 +1,8 @@
 # Switching a from-source install to OTP releases
 
-## What are OTP releases?
-OTP releases are as close as you can get to binary releases with Erlang/Elixir. The release is self-contained, and provides everything needed to boot it, it is easily administered via the provided shell script to open up a remote console, start/stop/restart the release, start in the background, send remote commands, and more.
+{! backend/installation/otp_vs_from_source.include !}
+
+In this guide we cover how you can migrate from a from source installation to one using OTP releases.
 
 ## Pre-requisites
 You will be running commands as root. If you aren't root already, please elevate your priviledges by executing `sudo su`/`su`.

docs/installation/otp_en.md

@@ -1,5 +1,9 @@
 # Installing on Linux using OTP releases
 
+{! backend/installation/otp_vs_from_source.include !}
+
+This guide covers a installation using an OTP release. To install Pleroma from source, please check out the corresponding guide for your distro.
+
 ## Pre-requisites
 * A machine running Linux with GNU (e.g. Debian, Ubuntu) or musl (e.g. Alpine) libc and `x86_64`, `aarch64` or `armv7l` CPU, you have root access to. If you are not sure if it's compatible see [Detecting flavour section](#detecting-flavour) below
 * A (sub)domain pointed to the machine

docs/installation/otp_vs_from_source.include

@@ -0,0 +1,3 @@
+## OTP releases vs from-source installations
+
+There are two ways to install Pleroma. You can use OTP releases or do a from-source installation. OTP releases are as close as you can get to binary releases with Erlang/Elixir. The release is self-contained, and provides everything needed to boot it, it is easily administered via the provided shell script to open up a remote console, start/stop/restart the release, start in the background, send remote commands, and more. With from source installations you install Pleroma from source, meaning you have to install certain dependencies like Erlang+Elixir and compile Pleroma yourself.

docs/installation/otp_vs_from_source_source.include

@@ -0,0 +1,3 @@
+{! backend/installation/otp_vs_from_source.include !}
+
+This guide covers a from-source installation. To install using OTP releases, please check out [the OTP guide](./otp_en.md).

lib/mix/tasks/pleroma/database.ex

@@ -209,7 +209,9 @@ defmodule Mix.Tasks.Pleroma.Database do
           new.fts_content := to_tsvector(new.data->>'content');
           RETURN new;
           END
-          $$ LANGUAGE plpgsql"
+          $$ LANGUAGE plpgsql",
+          [],
+          timeout: :infinity
         )
 
         shell_info("Refresh RUM index")
@@ -219,7 +221,9 @@ defmodule Mix.Tasks.Pleroma.Database do
 
         Ecto.Adapters.SQL.query!(
           Pleroma.Repo,
-          "CREATE INDEX objects_fts ON objects USING gin(to_tsvector('#{tsconfig}', data->>'content')); "
+          "CREATE INDEX CONCURRENTLY objects_fts ON objects USING gin(to_tsvector('#{tsconfig}', data->>'content')); ",
+          [],
+          timeout: :infinity
         )
       end
 

lib/pleroma/activity/search.ex

@@ -26,19 +26,23 @@ defmodule Pleroma.Activity.Search do
         :plain
       end
 
-    Activity
-    |> Activity.with_preloaded_object()
-    |> Activity.restrict_deactivated_users()
-    |> restrict_public()
-    |> query_with(index_type, search_query, search_function)
-    |> maybe_restrict_local(user)
-    |> maybe_restrict_author(author)
-    |> maybe_restrict_blocked(user)
-    |> Pagination.fetch_paginated(
-      %{"offset" => offset, "limit" => limit, "skip_order" => index_type == :rum},
-      :offset
-    )
-    |> maybe_fetch(user, search_query)
+    try do
+      Activity
+      |> Activity.with_preloaded_object()
+      |> Activity.restrict_deactivated_users()
+      |> restrict_public()
+      |> query_with(index_type, search_query, search_function)
+      |> maybe_restrict_local(user)
+      |> maybe_restrict_author(author)
+      |> maybe_restrict_blocked(user)
+      |> Pagination.fetch_paginated(
+        %{"offset" => offset, "limit" => limit, "skip_order" => index_type == :rum},
+        :offset
+      )
+      |> maybe_fetch(user, search_query)
+    rescue
+      _ -> maybe_fetch([], user, search_query)
+    end
   end
 
   def maybe_restrict_author(query, %User{} = author) do
@@ -61,10 +65,17 @@ defmodule Pleroma.Activity.Search do
   end
 
   defp query_with(q, :gin, search_query, :plain) do
+    %{rows: [[tsc]]} =
+      Ecto.Adapters.SQL.query!(
+        Pleroma.Repo,
+        "select current_setting('default_text_search_config')::regconfig::oid;"
+      )
+
     from([a, o] in q,
       where:
         fragment(
-          "to_tsvector(?->>'content') @@ plainto_tsquery(?)",
+          "to_tsvector(?::oid::regconfig, ?->>'content') @@ plainto_tsquery(?)",
+          ^tsc,
           o.data,
           ^search_query
         )
@@ -72,10 +83,17 @@ defmodule Pleroma.Activity.Search do
   end
 
   defp query_with(q, :gin, search_query, :websearch) do
+    %{rows: [[tsc]]} =
+      Ecto.Adapters.SQL.query!(
+        Pleroma.Repo,
+        "select current_setting('default_text_search_config')::regconfig::oid;"
+      )
+
     from([a, o] in q,
       where:
         fragment(
-          "to_tsvector(?->>'content') @@ websearch_to_tsquery(?)",
+          "to_tsvector(?::oid::regconfig, ?->>'content') @@ websearch_to_tsquery(?)",
+          ^tsc,
           o.data,
           ^search_query
         )

lib/pleroma/mfa/totp.ex

@@ -34,7 +34,7 @@ defmodule Pleroma.MFA.TOTP do
   defp default_digits, do: Config.get(@config_ns ++ [:digits])
 
   defp default_issuer,
-    do: Config.get(@config_ns ++ [:issuer], Config.get([:instance, :name]))
+    do: Config.get(@config_ns ++ [:issuer], Config.get([:instance, :host]))
 
   @doc "Creates a random Base 32 encoded string"
   def generate_secret do

lib/pleroma/web/activity_pub/mrf.ex

@@ -21,7 +21,7 @@ defmodule Pleroma.Web.ActivityPub.MRF do
           type: [:module, {:list, :module}],
           description:
             "A list of MRF policies enabled. Module names are shortened (removed leading `Pleroma.Web.ActivityPub.MRF.` part), but on adding custom module you need to use full name.",
-          suggestions: {:list_behaviour_implementations, Pleroma.Web.ActivityPub.MRF}
+          suggestions: {:list_behaviour_implementations, Pleroma.Web.ActivityPub.MRF.Policy}
         },
         %{
           key: :transparency,

lib/pleroma/web/activity_pub/mrf/object_age_policy.ex

@@ -49,6 +49,8 @@ defmodule Pleroma.Web.ActivityPub.MRF.ObjectAgePolicy do
           message
           |> Map.put("to", to)
           |> Map.put("cc", cc)
+          |> Kernel.put_in(["object", "to"], to)
+          |> Kernel.put_in(["object", "cc"], cc)
 
         {:ok, message}
       else
@@ -70,6 +72,8 @@ defmodule Pleroma.Web.ActivityPub.MRF.ObjectAgePolicy do
           message
           |> Map.put("to", to)
           |> Map.put("cc", cc)
+          |> Kernel.put_in(["object", "to"], to)
+          |> Kernel.put_in(["object", "cc"], cc)
 
         {:ok, message}
       else
@@ -82,7 +86,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.ObjectAgePolicy do
   end
 
   @impl true
-  def filter(%{"type" => "Create", "published" => _} = message) do
+  def filter(%{"type" => "Create", "object" => %{"published" => _}} = message) do
     with actions <- Config.get([:mrf_object_age, :actions]),
          {:reject, _} <- check_date(message),
          {:ok, message} <- check_reject(message, actions),

lib/pleroma/web/activity_pub/mrf/steal_emoji_policy.ex

@@ -93,6 +93,51 @@ defmodule Pleroma.Web.ActivityPub.MRF.StealEmojiPolicy do
   def filter(message), do: {:ok, message}
 
   @impl true
+  @spec config_description :: %{
+          children: [
+            %{
+              description: <<_::272, _::_*256>>,
+              key: :hosts | :rejected_shortcodes | :size_limit,
+              suggestions: [any(), ...],
+              type: {:list, :string} | {:list, :string} | :integer
+            },
+            ...
+          ],
+          description: <<_::448>>,
+          key: :mrf_steal_emoji,
+          label: <<_::80>>,
+          related_policy: <<_::352>>
+        }
+  def config_description do
+    %{
+      key: :mrf_steal_emoji,
+      related_policy: "Pleroma.Web.ActivityPub.MRF.StealEmojiPolicy",
+      label: "MRF Emojis",
+      description: "Steals emojis from selected instances when it sees them.",
+      children: [
+        %{
+          key: :hosts,
+          type: {:list, :string},
+          description: "List of hosts to steal emojis from",
+          suggestions: [""]
+        },
+        %{
+          key: :rejected_shortcodes,
+          type: {:list, :string},
+          description: "Regex-list of shortcodes to reject",
+          suggestions: [""]
+        },
+        %{
+          key: :size_limit,
+          type: :integer,
+          description: "File size limit (in bytes), checked before an emoji is saved to the disk",
+          suggestions: ["100000"]
+        }
+      ]
+    }
+  end
+
+  @impl true
   def describe do
     {:ok, %{}}
   end

lib/pleroma/web/activity_pub/side_effects.ex

@@ -10,7 +10,6 @@ defmodule Pleroma.Web.ActivityPub.SideEffects do
   collection, and so on.
   """
   alias Pleroma.Activity
-  alias Pleroma.Activity.Ir.Topics
   alias Pleroma.Chat
   alias Pleroma.Chat.MessageReference
   alias Pleroma.FollowingRelationship
@@ -225,6 +224,8 @@ defmodule Pleroma.Web.ActivityPub.SideEffects do
         meta
         |> add_notifications(notifications)
 
+      ap_streamer().stream_out(activity)
+
       {:ok, activity, meta}
     else
       e -> Repo.rollback(e)
@@ -245,9 +246,7 @@ defmodule Pleroma.Web.ActivityPub.SideEffects do
     if !User.is_internal_user?(user) do
       Notification.create_notifications(object)
 
-      object
-      |> Topics.get_activity_topics()
-      |> Streamer.stream(object)
+      ap_streamer().stream_out(object)
     end
 
     {:ok, object, meta}

lib/pleroma/web/admin_api/controllers/frontend_controller.ex

@@ -35,6 +35,12 @@ defmodule Pleroma.Web.AdminAPI.FrontendController do
   end
 
   defp installed do
-    File.ls!(Pleroma.Frontend.dir())
+    frontend_directory = Pleroma.Frontend.dir()
+
+    if File.exists?(frontend_directory) do
+      File.ls!(frontend_directory)
+    else
+      []
+    end
   end
 end

lib/pleroma/web/admin_api/report.ex

@@ -13,7 +13,9 @@ defmodule Pleroma.Web.AdminAPI.Report do
     account = User.get_cached_by_ap_id(account_ap_id)
 
     statuses =
-      Enum.map(status_ap_ids, fn
+      status_ap_ids
+      |> Enum.reject(&is_nil(&1))
+      |> Enum.map(fn
         act when is_map(act) -> Activity.get_by_ap_id_with_object(act["id"])
         act when is_binary(act) -> Activity.get_by_ap_id_with_object(act)
       end)

lib/pleroma/web/api_spec/operations/twitter_util_operation.ex

@@ -8,6 +8,8 @@ defmodule Pleroma.Web.ApiSpec.TwitterUtilOperation do
   alias Pleroma.Web.ApiSpec.Schemas.ApiError
   alias Pleroma.Web.ApiSpec.Schemas.BooleanLike
 
+  import Pleroma.Web.ApiSpec.Helpers
+
   def open_api_operation(action) do
     operation = String.to_existing_atom("#{action}_operation")
     apply(__MODULE__, operation, [])
@@ -63,17 +65,7 @@ defmodule Pleroma.Web.ApiSpec.TwitterUtilOperation do
       summary: "Change account password",
       security: [%{"oAuth" => ["write:accounts"]}],
       operationId: "UtilController.change_password",
-      parameters: [
-        Operation.parameter(:password, :query, :string, "Current password", required: true),
-        Operation.parameter(:new_password, :query, :string, "New password", required: true),
-        Operation.parameter(
-          :new_password_confirmation,
-          :query,
-          :string,
-          "New password, confirmation",
-          required: true
-        )
-      ],
+      requestBody: request_body("Parameters", change_password_request(), required: true),
       responses: %{
         200 =>
           Operation.response("Success", "application/json", %Schema{
@@ -86,17 +78,30 @@ defmodule Pleroma.Web.ApiSpec.TwitterUtilOperation do
     }
   end
 
+  defp change_password_request do
+    %Schema{
+      title: "ChangePasswordRequest",
+      description: "POST body for changing the account's passowrd",
+      type: :object,
+      required: [:password, :new_password, :new_password_confirmation],
+      properties: %{
+        password: %Schema{type: :string, description: "Current password"},
+        new_password: %Schema{type: :string, description: "New password"},
+        new_password_confirmation: %Schema{
+          type: :string,
+          description: "New password, confirmation"
+        }
+      }
+    }
+  end
+
   def change_email_operation do
     %Operation{
       tags: ["Account credentials"],
       summary: "Change account email",
       security: [%{"oAuth" => ["write:accounts"]}],
       operationId: "UtilController.change_email",
-      parameters: [
-        Operation.parameter(:password, :query, :string, "Current password", required: true),
-        Operation.parameter(:email, :query, :string, "New email", required: true)
-      ],
-      requestBody: nil,
+      requestBody: request_body("Parameters", change_email_request(), required: true),
       responses: %{
         200 =>
           Operation.response("Success", "application/json", %Schema{
@@ -109,6 +114,19 @@ defmodule Pleroma.Web.ApiSpec.TwitterUtilOperation do
     }
   end
 
+  defp change_email_request do
+    %Schema{
+      title: "ChangeEmailRequest",
+      description: "POST body for changing the account's email",
+      type: :object,
+      required: [:email, :password],
+      properties: %{
+        email: %Schema{type: :string, description: "New email"},
+        password: %Schema{type: :string, description: "Current password"}
+      }
+    }
+  end
+
   def update_notificaton_settings_operation do
     %Operation{
       tags: ["Accounts"],

lib/pleroma/web/common_api/utils.ex

@@ -412,19 +412,14 @@ defmodule Pleroma.Web.CommonAPI.Utils do
 
   def maybe_notify_mentioned_recipients(recipients, _), do: recipients
 
-  # Do not notify subscribers if author is making a reply
-  def maybe_notify_subscribers(recipients, %Activity{
-        object: %Object{data: %{"inReplyTo" => _ap_id}}
-      }) do
-    recipients
-  end
-
   def maybe_notify_subscribers(
         recipients,
-        %Activity{data: %{"actor" => actor, "type" => type}} = activity
-      )
-      when type == "Create" do
-    with %User{} = user <- User.get_cached_by_ap_id(actor) do
+        %Activity{data: %{"actor" => actor, "type" => "Create"}} = activity
+      ) do
+    # Do not notify subscribers if author is making a reply
+    with %Object{data: object} <- Object.normalize(activity, fetch: false),
+         nil <- object["inReplyTo"],
+         %User{} = user <- User.get_cached_by_ap_id(actor) do
       subscriber_ids =
         user
         |> User.subscriber_users()

lib/pleroma/web/mastodon_api/views/status_view.ex

@@ -65,11 +65,19 @@ defmodule Pleroma.Web.MastodonAPI.StatusView do
 
   defp get_context_id(_), do: nil
 
-  defp reblogged?(activity, user) do
-    object = Object.normalize(activity, fetch: false) || %{}
-    present?(user && user.ap_id in (object.data["announcements"] || []))
+  # Check if the user reblogged this status
+  defp reblogged?(activity, %User{ap_id: ap_id}) do
+    with %Object{data: %{"announcements" => announcements}} when is_list(announcements) <-
+           Object.normalize(activity, fetch: false) do
+      ap_id in announcements
+    else
+      _ -> false
+    end
   end
 
+  # False if the user is logged out
+  defp reblogged?(_activity, _user), do: false
+
   def render("index.json", opts) do
     reading_user = opts[:for]
 

lib/pleroma/web/pleroma_api/controllers/two_factor_authentication_controller.ex

@@ -41,7 +41,7 @@ defmodule Pleroma.Web.PleromaAPI.TwoFactorAuthenticationController do
   def setup(%{assigns: %{user: user}} = conn, %{"method" => "totp"} = _params) do
     with {:ok, user} <- MFA.setup_totp(user),
          %{secret: secret} = _ <- user.multi_factor_authentication_settings.totp do
-      provisioning_uri = TOTP.provisioning_uri(secret, "#{user.email}")
+      provisioning_uri = TOTP.provisioning_uri(secret, "#{user.ap_id}")
 
       json(conn, %{provisioning_uri: provisioning_uri, key: secret})
     else

lib/pleroma/web/twitter_api/controllers/util_controller.ex

@@ -81,17 +81,13 @@ defmodule Pleroma.Web.TwitterAPI.UtilController do
     end
   end
 
-  def change_password(%{assigns: %{user: user}} = conn, %{
-        password: password,
-        new_password: new_password,
-        new_password_confirmation: new_password_confirmation
-      }) do
-    case CommonAPI.Utils.confirm_current_password(user, password) do
+  def change_password(%{assigns: %{user: user}, body_params: body_params} = conn, %{}) do
+    case CommonAPI.Utils.confirm_current_password(user, body_params.password) do
       {:ok, user} ->
         with {:ok, _user} <-
                User.reset_password(user, %{
-                 password: new_password,
-                 password_confirmation: new_password_confirmation
+                 password: body_params.new_password,
+                 password_confirmation: body_params.new_password_confirmation
                }) do
           json(conn, %{status: "success"})
         else
@@ -108,10 +104,10 @@ defmodule Pleroma.Web.TwitterAPI.UtilController do
     end
   end
 
-  def change_email(%{assigns: %{user: user}} = conn, %{password: password, email: email}) do
-    case CommonAPI.Utils.confirm_current_password(user, password) do
+  def change_email(%{assigns: %{user: user}, body_params: body_params} = conn, %{}) do
+    case CommonAPI.Utils.confirm_current_password(user, body_params.password) do
       {:ok, user} ->
-        with {:ok, _user} <- User.change_email(user, email) do
+        with {:ok, _user} <- User.change_email(user, body_params.email) do
           json(conn, %{status: "success"})
         else
           {:error, changeset} ->

mix.exs

@@ -4,7 +4,7 @@ defmodule Pleroma.Mixfile do
   def project do
     [
       app: :pleroma,
-      version: version("2.4.0"),
+      version: version("2.4.1"),
       elixir: "~> 1.9",
       elixirc_paths: elixirc_paths(Mix.env()),
       compilers: [:phoenix, :gettext] ++ Mix.compilers(),

test/pleroma/docs/generator_test.exs

@@ -23,7 +23,7 @@ defmodule Pleroma.Docs.GeneratorTest do
           key: :filters,
           type: {:list, :module},
           description: "",
-          suggestions: {:list_behaviour_implementations, Pleroma.Web.ActivityPub.MRF}
+          suggestions: {:list_behaviour_implementations, Pleroma.Web.ActivityPub.MRF.Policy}
         },
         %{
           key: Pleroma.Upload,

test/pleroma/web/activity_pub/mrf/object_age_policy_test.exs

@@ -22,6 +22,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.ObjectAgePolicyTest do
   defp get_old_message do
     File.read!("test/fixtures/mastodon-post-activity.json")
     |> Jason.decode!()
+    |> Map.drop(["published"])
   end
 
   defp get_new_message do

test/pleroma/web/admin_api/controllers/frontend_controller_test.exs

@@ -42,6 +42,20 @@ defmodule Pleroma.Web.AdminAPI.FrontendControllerTest do
 
       refute Enum.any?(response, fn frontend -> frontend["installed"] == true end)
     end
+
+    test "it lists available frontends when no frontend folder was created yet", %{conn: conn} do
+      File.rm_rf(@dir)
+
+      response =
+        conn
+        |> get("/api/pleroma/admin/frontends")
+        |> json_response_and_validate_schema(:ok)
+
+      assert Enum.map(response, & &1["name"]) ==
+               Enum.map(Config.get([:frontends, :available]), fn {_, map} -> map["name"] end)
+
+      refute Enum.any?(response, fn frontend -> frontend["installed"] == true end)
+    end
   end
 
   describe "POST /api/pleroma/admin/frontends/install" do

test/pleroma/web/twitter_api/util_controller_test.exs

@@ -261,11 +261,8 @@ defmodule Pleroma.Web.TwitterAPI.UtilControllerTest do
       conn =
         conn
         |> assign(:token, nil)
-        |> post(
-          "/api/pleroma/change_email?#{
-            URI.encode_query(%{password: "hi", email: "test@test.com"})
-          }"
-        )
+        |> put_req_header("content-type", "multipart/form-data")
+        |> post("/api/pleroma/change_email", %{password: "hi", email: "test@test.com"})
 
       assert json_response_and_validate_schema(conn, 403) == %{
                "error" => "Insufficient permissions: write:accounts."
@@ -274,12 +271,9 @@ defmodule Pleroma.Web.TwitterAPI.UtilControllerTest do
 
     test "with proper permissions and invalid password", %{conn: conn} do
       conn =
-        post(
-          conn,
-          "/api/pleroma/change_email?#{
-            URI.encode_query(%{password: "hi", email: "test@test.com"})
-          }"
-        )
+        conn
+        |> put_req_header("content-type", "multipart/form-data")
+        |> post("/api/pleroma/change_email", %{password: "hi", email: "test@test.com"})
 
       assert json_response_and_validate_schema(conn, 200) == %{"error" => "Invalid password."}
     end
@@ -288,10 +282,9 @@ defmodule Pleroma.Web.TwitterAPI.UtilControllerTest do
       conn: conn
     } do
       conn =
-        post(
-          conn,
-          "/api/pleroma/change_email?#{URI.encode_query(%{password: "test", email: "foobar"})}"
-        )
+        conn
+        |> put_req_header("content-type", "multipart/form-data")
+        |> post("/api/pleroma/change_email", %{password: "test", email: "foobar"})
 
       assert json_response_and_validate_schema(conn, 200) == %{
                "error" => "Email has invalid format."
@@ -301,7 +294,10 @@ defmodule Pleroma.Web.TwitterAPI.UtilControllerTest do
     test "with proper permissions, valid password and no email", %{
       conn: conn
     } do
-      conn = post(conn, "/api/pleroma/change_email?#{URI.encode_query(%{password: "test"})}")
+      conn =
+        conn
+        |> put_req_header("content-type", "multipart/form-data")
+        |> post("/api/pleroma/change_email", %{password: "test"})
 
       assert %{"error" => "Missing field: email."} = json_response_and_validate_schema(conn, 400)
     end
@@ -310,10 +306,9 @@ defmodule Pleroma.Web.TwitterAPI.UtilControllerTest do
       conn: conn
     } do
       conn =
-        post(
-          conn,
-          "/api/pleroma/change_email?#{URI.encode_query(%{password: "test", email: ""})}"
-        )
+        conn
+        |> put_req_header("content-type", "multipart/form-data")
+        |> post("/api/pleroma/change_email", %{password: "test", email: ""})
 
       assert json_response_and_validate_schema(conn, 200) == %{"error" => "Email can't be blank."}
     end
@@ -324,10 +319,9 @@ defmodule Pleroma.Web.TwitterAPI.UtilControllerTest do
       user = insert(:user)
 
       conn =
-        post(
-          conn,
-          "/api/pleroma/change_email?#{URI.encode_query(%{password: "test", email: user.email})}"
-        )
+        conn
+        |> put_req_header("content-type", "multipart/form-data")
+        |> post("/api/pleroma/change_email", %{password: "test", email: user.email})
 
       assert json_response_and_validate_schema(conn, 200) == %{
                "error" => "Email has already been taken."
@@ -338,12 +332,9 @@ defmodule Pleroma.Web.TwitterAPI.UtilControllerTest do
       conn: conn
     } do
       conn =
-        post(
-          conn,
-          "/api/pleroma/change_email?#{
-            URI.encode_query(%{password: "test", email: "cofe@foobar.com"})
-          }"
-        )
+        conn
+        |> put_req_header("content-type", "multipart/form-data")
+        |> post("/api/pleroma/change_email", %{password: "test", email: "cofe@foobar.com"})
 
       assert json_response_and_validate_schema(conn, 200) == %{"status" => "success"}
     end
@@ -356,15 +347,12 @@ defmodule Pleroma.Web.TwitterAPI.UtilControllerTest do
       conn =
         conn
         |> assign(:token, nil)
-        |> post(
-          "/api/pleroma/change_password?#{
-            URI.encode_query(%{
-              password: "hi",
-              new_password: "newpass",
-              new_password_confirmation: "newpass"
-            })
-          }"
-        )
+        |> put_req_header("content-type", "multipart/form-data")
+        |> post("/api/pleroma/change_password", %{
+          "password" => "hi",
+          "new_password" => "newpass",
+          "new_password_confirmation" => "newpass"
+        })
 
       assert json_response_and_validate_schema(conn, 403) == %{
                "error" => "Insufficient permissions: write:accounts."
@@ -373,16 +361,13 @@ defmodule Pleroma.Web.TwitterAPI.UtilControllerTest do
 
     test "with proper permissions and invalid password", %{conn: conn} do
       conn =
-        post(
-          conn,
-          "/api/pleroma/change_password?#{
-            URI.encode_query(%{
-              password: "hi",
-              new_password: "newpass",
-              new_password_confirmation: "newpass"
-            })
-          }"
-        )
+        conn
+        |> put_req_header("content-type", "multipart/form-data")
+        |> post("/api/pleroma/change_password", %{
+          "password" => "hi",
+          "new_password" => "newpass",
+          "new_password_confirmation" => "newpass"
+        })
 
       assert json_response_and_validate_schema(conn, 200) == %{"error" => "Invalid password."}
     end
@@ -392,16 +377,13 @@ defmodule Pleroma.Web.TwitterAPI.UtilControllerTest do
            conn: conn
          } do
       conn =
-        post(
-          conn,
-          "/api/pleroma/change_password?#{
-            URI.encode_query(%{
-              password: "test",
-              new_password: "newpass",
-              new_password_confirmation: "notnewpass"
-            })
-          }"
-        )
+        conn
+        |> put_req_header("content-type", "multipart/form-data")
+        |> post("/api/pleroma/change_password", %{
+          "password" => "test",
+          "new_password" => "newpass",
+          "new_password_confirmation" => "notnewpass"
+        })
 
       assert json_response_and_validate_schema(conn, 200) == %{
                "error" => "New password does not match confirmation."
@@ -412,12 +394,13 @@ defmodule Pleroma.Web.TwitterAPI.UtilControllerTest do
       conn: conn
     } do
       conn =
-        post(
-          conn,
-          "/api/pleroma/change_password?#{
-            URI.encode_query(%{password: "test", new_password: "", new_password_confirmation: ""})
-          }"
-        )
+        conn
+        |> put_req_header("content-type", "multipart/form-data")
+        |> post("/api/pleroma/change_password", %{
+          password: "test",
+          new_password: "",
+          new_password_confirmation: ""
+        })
 
       assert json_response_and_validate_schema(conn, 200) == %{
                "error" => "New password can't be blank."
@@ -429,15 +412,15 @@ defmodule Pleroma.Web.TwitterAPI.UtilControllerTest do
       user: user
     } do
       conn =
-        post(
-          conn,
-          "/api/pleroma/change_password?#{
-            URI.encode_query(%{
-              password: "test",
-              new_password: "newpass",
-              new_password_confirmation: "newpass"
-            })
-          }"
+        conn
+        |> put_req_header("content-type", "multipart/form-data")
+        |> post(
+          "/api/pleroma/change_password",
+          %{
+            password: "test",
+            new_password: "newpass",
+            new_password_confirmation: "newpass"
+          }
         )
 
       assert json_response_and_validate_schema(conn, 200) == %{"status" => "success"}