Merge branch 'global-status-expiration' into 'develop'

Global status expiration

See merge request pleroma/pleroma!2208
This commit is contained in:
lain 2020-06-12 14:14:09 +00:00
commit e557265a03
12 changed files with 218 additions and 22 deletions

View File

@ -6,6 +6,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## [unreleased] ## [unreleased]
### Changed ### Changed
- MFR policy to set global expiration for all local Create activities
<details> <details>
<summary>API Changes</summary> <summary>API Changes</summary>
- **Breaking:** Emoji API: changed methods and renamed routes. - **Breaking:** Emoji API: changed methods and renamed routes.

View File

@ -371,6 +371,8 @@ config :pleroma, :mrf_keyword,
config :pleroma, :mrf_subchain, match_actor: %{} config :pleroma, :mrf_subchain, match_actor: %{}
config :pleroma, :mrf_activity_expiration, days: 365
config :pleroma, :mrf_vocabulary, config :pleroma, :mrf_vocabulary,
accept: [], accept: [],
reject: [] reject: []

View File

@ -1473,6 +1473,21 @@ config :pleroma, :config_description, [
}, },
%{ %{
group: :pleroma, group: :pleroma,
key: :mrf_activity_expiration,
label: "MRF Activity Expiration Policy",
type: :group,
description: "Adds expiration to all local Create Note activities",
children: [
%{
key: :days,
type: :integer,
description: "Default global expiration time for all local Create activities (in days)",
suggestions: [90, 365]
}
]
},
%{
group: :pleroma,
key: :mrf_subchain, key: :mrf_subchain,
label: "MRF subchain", label: "MRF subchain",
type: :group, type: :group,

View File

@ -39,7 +39,7 @@ To add configuration to your config file, you can copy it from the base config.
* `rewrite_policy`: Message Rewrite Policy, either one or a list. Here are the ones available by default: * `rewrite_policy`: Message Rewrite Policy, either one or a list. Here are the ones available by default:
* `Pleroma.Web.ActivityPub.MRF.NoOpPolicy`: Doesnt modify activities (default). * `Pleroma.Web.ActivityPub.MRF.NoOpPolicy`: Doesnt modify activities (default).
* `Pleroma.Web.ActivityPub.MRF.DropPolicy`: Drops all activities. It generally doesnt makes sense to use in production. * `Pleroma.Web.ActivityPub.MRF.DropPolicy`: Drops all activities. It generally doesnt makes sense to use in production.
* `Pleroma.Web.ActivityPub.MRF.SimplePolicy`: Restrict the visibility of activities from certains instances (See [`:mrf_simple`](#mrf_simple)). * `Pleroma.Web.ActivityPub.MRF.SimplePolicy`: Restrict the visibility of activities from certain instances (See [`:mrf_simple`](#mrf_simple)).
* `Pleroma.Web.ActivityPub.MRF.TagPolicy`: Applies policies to individual users based on tags, which can be set using pleroma-fe/admin-fe/any other app that supports Pleroma Admin API. For example it allows marking posts from individual users nsfw (sensitive). * `Pleroma.Web.ActivityPub.MRF.TagPolicy`: Applies policies to individual users based on tags, which can be set using pleroma-fe/admin-fe/any other app that supports Pleroma Admin API. For example it allows marking posts from individual users nsfw (sensitive).
* `Pleroma.Web.ActivityPub.MRF.SubchainPolicy`: Selectively runs other MRF policies when messages match (See [`:mrf_subchain`](#mrf_subchain)). * `Pleroma.Web.ActivityPub.MRF.SubchainPolicy`: Selectively runs other MRF policies when messages match (See [`:mrf_subchain`](#mrf_subchain)).
* `Pleroma.Web.ActivityPub.MRF.RejectNonPublic`: Drops posts with non-public visibility settings (See [`:mrf_rejectnonpublic`](#mrf_rejectnonpublic)). * `Pleroma.Web.ActivityPub.MRF.RejectNonPublic`: Drops posts with non-public visibility settings (See [`:mrf_rejectnonpublic`](#mrf_rejectnonpublic)).
@ -49,7 +49,8 @@ To add configuration to your config file, you can copy it from the base config.
* `Pleroma.Web.ActivityPub.MRF.MentionPolicy`: Drops posts mentioning configurable users. (See [`:mrf_mention`](#mrf_mention)). * `Pleroma.Web.ActivityPub.MRF.MentionPolicy`: Drops posts mentioning configurable users. (See [`:mrf_mention`](#mrf_mention)).
* `Pleroma.Web.ActivityPub.MRF.VocabularyPolicy`: Restricts activities to a configured set of vocabulary. (See [`:mrf_vocabulary`](#mrf_vocabulary)). * `Pleroma.Web.ActivityPub.MRF.VocabularyPolicy`: Restricts activities to a configured set of vocabulary. (See [`:mrf_vocabulary`](#mrf_vocabulary)).
* `Pleroma.Web.ActivityPub.MRF.ObjectAgePolicy`: Rejects or delists posts based on their age when received. (See [`:mrf_object_age`](#mrf_object_age)). * `Pleroma.Web.ActivityPub.MRF.ObjectAgePolicy`: Rejects or delists posts based on their age when received. (See [`:mrf_object_age`](#mrf_object_age)).
* `public`: Makes the client API in authentificated mode-only except for user-profiles. Useful for disabling the Local Timeline and The Whole Known Network. * `Pleroma.Web.ActivityPub.MRF.ActivityExpirationPolicy`: Adds expiration to all local Create activities (see [`:mrf_activity_expiration`](#mrf_activity_expiration)).
* `public`: Makes the client API in authenticated mode-only except for user-profiles. Useful for disabling the Local Timeline and The Whole Known Network.
* `quarantined_instances`: List of ActivityPub instances where private(DMs, followers-only) activities will not be send. * `quarantined_instances`: List of ActivityPub instances where private(DMs, followers-only) activities will not be send.
* `managed_config`: Whenether the config for pleroma-fe is configured in [:frontend_configurations](#frontend_configurations) or in ``static/config.json``. * `managed_config`: Whenether the config for pleroma-fe is configured in [:frontend_configurations](#frontend_configurations) or in ``static/config.json``.
* `allowed_post_formats`: MIME-type list of formats allowed to be posted (transformed into HTML). * `allowed_post_formats`: MIME-type list of formats allowed to be posted (transformed into HTML).
@ -154,6 +155,10 @@ config :pleroma, :mrf_user_allowlist,
* `rejected_shortcodes`: Regex-list of shortcodes to reject * `rejected_shortcodes`: Regex-list of shortcodes to reject
* `size_limit`: File size limit (in bytes), checked before an emoji is saved to the disk * `size_limit`: File size limit (in bytes), checked before an emoji is saved to the disk
#### :mrf_activity_expiration
* `days`: Default global expiration time for all local Create activities (in days)
### :activitypub ### :activitypub
* `unfollow_blocked`: Whether blocks result in people getting unfollowed * `unfollow_blocked`: Whether blocks result in people getting unfollowed
* `outgoing_blocks`: Whether to federate blocks to other instances * `outgoing_blocks`: Whether to federate blocks to other instances

View File

@ -5,6 +5,7 @@
defmodule Pleroma.Web.ActivityPub.ActivityPub do defmodule Pleroma.Web.ActivityPub.ActivityPub do
alias Pleroma.Activity alias Pleroma.Activity
alias Pleroma.Activity.Ir.Topics alias Pleroma.Activity.Ir.Topics
alias Pleroma.ActivityExpiration
alias Pleroma.Config alias Pleroma.Config
alias Pleroma.Constants alias Pleroma.Constants
alias Pleroma.Conversation alias Pleroma.Conversation
@ -146,12 +147,14 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
{:containment, :ok} <- {:containment, Containment.contain_child(map)}, {:containment, :ok} <- {:containment, Containment.contain_child(map)},
{:ok, map, object} <- insert_full_object(map) do {:ok, map, object} <- insert_full_object(map) do
{:ok, activity} = {:ok, activity} =
Repo.insert(%Activity{ %Activity{
data: map, data: map,
local: local, local: local,
actor: map["actor"], actor: map["actor"],
recipients: recipients recipients: recipients
}) }
|> Repo.insert()
|> maybe_create_activity_expiration()
# Splice in the child object if we have one. # Splice in the child object if we have one.
activity = Maps.put_if_present(activity, :object, object) activity = Maps.put_if_present(activity, :object, object)
@ -189,6 +192,14 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
stream_out_participations(participations) stream_out_participations(participations)
end end
defp maybe_create_activity_expiration({:ok, %{data: %{"expires_at" => expires_at}} = activity}) do
with {:ok, _} <- ActivityExpiration.create(activity, expires_at) do
{:ok, activity}
end
end
defp maybe_create_activity_expiration(result), do: result
defp create_or_bump_conversation(activity, actor) do defp create_or_bump_conversation(activity, actor) do
with {:ok, conversation} <- Conversation.create_or_bump_for(activity), with {:ok, conversation} <- Conversation.create_or_bump_for(activity),
%User{} = user <- User.get_cached_by_ap_id(actor) do %User{} = user <- User.get_cached_by_ap_id(actor) do

View File

@ -8,11 +8,8 @@ defmodule Pleroma.Web.ActivityPub.MRF do
def filter(policies, %{} = object) do def filter(policies, %{} = object) do
policies policies
|> Enum.reduce({:ok, object}, fn |> Enum.reduce({:ok, object}, fn
policy, {:ok, object} -> policy, {:ok, object} -> policy.filter(object)
policy.filter(object) _, error -> error
_, error ->
error
end) end)
end end

View File

@ -0,0 +1,43 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.ActivityPub.MRF.ActivityExpirationPolicy do
@moduledoc "Adds expiration to all local Create activities"
@behaviour Pleroma.Web.ActivityPub.MRF
@impl true
def filter(activity) do
activity =
if note?(activity) and local?(activity) do
maybe_add_expiration(activity)
else
activity
end
{:ok, activity}
end
@impl true
def describe, do: {:ok, %{}}
defp local?(%{"id" => id}) do
String.starts_with?(id, Pleroma.Web.Endpoint.url())
end
defp note?(activity) do
match?(%{"type" => "Create", "object" => %{"type" => "Note"}}, activity)
end
defp maybe_add_expiration(activity) do
days = Pleroma.Config.get([:mrf_activity_expiration, :days], 365)
expires_at = NaiveDateTime.utc_now() |> Timex.shift(days: days)
with %{"expires_at" => existing_expires_at} <- activity,
:lt <- NaiveDateTime.compare(existing_expires_at, expires_at) do
activity
else
_ -> Map.put(activity, "expires_at", expires_at)
end
end
end

View File

@ -197,6 +197,13 @@ defmodule Pleroma.Web.CommonAPI.ActivityDraft do
defp changes(draft) do defp changes(draft) do
direct? = draft.visibility == "direct" direct? = draft.visibility == "direct"
additional = %{"cc" => draft.cc, "directMessage" => direct?}
additional =
case draft.expires_at do
%NaiveDateTime{} = expires_at -> Map.put(additional, "expires_at", expires_at)
_ -> additional
end
changes = changes =
%{ %{
@ -204,7 +211,7 @@ defmodule Pleroma.Web.CommonAPI.ActivityDraft do
actor: draft.user, actor: draft.user,
context: draft.context, context: draft.context,
object: draft.object, object: draft.object,
additional: %{"cc" => draft.cc, "directMessage" => direct?} additional: additional
} }
|> Utils.maybe_add_list_data(draft.user, draft.visibility) |> Utils.maybe_add_list_data(draft.user, draft.visibility)

View File

@ -423,20 +423,10 @@ defmodule Pleroma.Web.CommonAPI do
def post(user, %{status: _} = data) do def post(user, %{status: _} = data) do
with {:ok, draft} <- Pleroma.Web.CommonAPI.ActivityDraft.create(user, data) do with {:ok, draft} <- Pleroma.Web.CommonAPI.ActivityDraft.create(user, data) do
draft.changes ActivityPub.create(draft.changes, draft.preview?)
|> ActivityPub.create(draft.preview?)
|> maybe_create_activity_expiration(draft.expires_at)
end end
end end
defp maybe_create_activity_expiration({:ok, activity}, %NaiveDateTime{} = expires_at) do
with {:ok, _} <- ActivityExpiration.create(activity, expires_at) do
{:ok, activity}
end
end
defp maybe_create_activity_expiration(result, _), do: result
def pin(id, %{ap_id: user_ap_id} = user) do def pin(id, %{ap_id: user_ap_id} = user) do
with %Activity{ with %Activity{
actor: ^user_ap_id, actor: ^user_ap_id,

View File

@ -1986,4 +1986,20 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubTest do
end) =~ "Follower/Following counter update for #{user.ap_id} failed" end) =~ "Follower/Following counter update for #{user.ap_id} failed"
end end
end end
describe "global activity expiration" do
setup do: clear_config([:instance, :rewrite_policy])
test "creates an activity expiration for local Create activities" do
Pleroma.Config.put(
[:instance, :rewrite_policy],
Pleroma.Web.ActivityPub.MRF.ActivityExpirationPolicy
)
{:ok, %{id: id_create}} = ActivityBuilder.insert(%{"type" => "Create", "context" => "3hu"})
{:ok, _follow} = ActivityBuilder.insert(%{"type" => "Follow", "context" => "3hu"})
assert [%{activity_id: ^id_create}] = Pleroma.ActivityExpiration |> Repo.all()
end
end
end end

View File

@ -0,0 +1,77 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.ActivityPub.MRF.ActivityExpirationPolicyTest do
use ExUnit.Case, async: true
alias Pleroma.Web.ActivityPub.MRF.ActivityExpirationPolicy
@id Pleroma.Web.Endpoint.url() <> "/activities/cofe"
test "adds `expires_at` property" do
assert {:ok, %{"type" => "Create", "expires_at" => expires_at}} =
ActivityExpirationPolicy.filter(%{
"id" => @id,
"type" => "Create",
"object" => %{"type" => "Note"}
})
assert Timex.diff(expires_at, NaiveDateTime.utc_now(), :days) == 364
end
test "keeps existing `expires_at` if it less than the config setting" do
expires_at = NaiveDateTime.utc_now() |> Timex.shift(days: 1)
assert {:ok, %{"type" => "Create", "expires_at" => ^expires_at}} =
ActivityExpirationPolicy.filter(%{
"id" => @id,
"type" => "Create",
"expires_at" => expires_at,
"object" => %{"type" => "Note"}
})
end
test "overwrites existing `expires_at` if it greater than the config setting" do
too_distant_future = NaiveDateTime.utc_now() |> Timex.shift(years: 2)
assert {:ok, %{"type" => "Create", "expires_at" => expires_at}} =
ActivityExpirationPolicy.filter(%{
"id" => @id,
"type" => "Create",
"expires_at" => too_distant_future,
"object" => %{"type" => "Note"}
})
assert Timex.diff(expires_at, NaiveDateTime.utc_now(), :days) == 364
end
test "ignores remote activities" do
assert {:ok, activity} =
ActivityExpirationPolicy.filter(%{
"id" => "https://example.com/123",
"type" => "Create",
"object" => %{"type" => "Note"}
})
refute Map.has_key?(activity, "expires_at")
end
test "ignores non-Create/Note activities" do
assert {:ok, activity} =
ActivityExpirationPolicy.filter(%{
"id" => "https://example.com/123",
"type" => "Follow"
})
refute Map.has_key?(activity, "expires_at")
assert {:ok, activity} =
ActivityExpirationPolicy.filter(%{
"id" => "https://example.com/123",
"type" => "Create",
"object" => %{"type" => "Cofe"}
})
refute Map.has_key?(activity, "expires_at")
end
end

View File

@ -11,7 +11,10 @@ defmodule Pleroma.Workers.Cron.PurgeExpiredActivitiesWorkerTest do
import Pleroma.Factory import Pleroma.Factory
import ExUnit.CaptureLog import ExUnit.CaptureLog
setup do: clear_config([ActivityExpiration, :enabled]) setup do
clear_config([ActivityExpiration, :enabled])
clear_config([:instance, :rewrite_policy])
end
test "deletes an expiration activity" do test "deletes an expiration activity" do
Pleroma.Config.put([ActivityExpiration, :enabled], true) Pleroma.Config.put([ActivityExpiration, :enabled], true)
@ -36,6 +39,35 @@ defmodule Pleroma.Workers.Cron.PurgeExpiredActivitiesWorkerTest do
refute Pleroma.Repo.get(Pleroma.ActivityExpiration, expiration.id) refute Pleroma.Repo.get(Pleroma.ActivityExpiration, expiration.id)
end end
test "works with ActivityExpirationPolicy" do
Pleroma.Config.put([ActivityExpiration, :enabled], true)
Pleroma.Config.put(
[:instance, :rewrite_policy],
Pleroma.Web.ActivityPub.MRF.ActivityExpirationPolicy
)
user = insert(:user)
days = Pleroma.Config.get([:mrf_activity_expiration, :days], 365)
{:ok, %{id: id} = activity} = Pleroma.Web.CommonAPI.post(user, %{status: "cofe"})
past_date =
NaiveDateTime.utc_now() |> Timex.shift(days: -days) |> NaiveDateTime.truncate(:second)
activity
|> Repo.preload(:expiration)
|> Map.get(:expiration)
|> Ecto.Changeset.change(%{scheduled_at: past_date})
|> Repo.update!()
Pleroma.Workers.Cron.PurgeExpiredActivitiesWorker.perform(:ops, :pid)
assert [%{data: %{"type" => "Delete", "deleted_activity_id" => ^id}}] =
Pleroma.Repo.all(Pleroma.Activity)
end
describe "delete_activity/1" do describe "delete_activity/1" do
test "adds log message if activity isn't find" do test "adds log message if activity isn't find" do
assert capture_log([level: :error], fn -> assert capture_log([level: :error], fn ->