Commit Graph

2682 Commits

Author SHA1 Message Date
Rhea Danzey a0327fd1b8 Allow templating to succeed if postgres deployment is enabled but a connection string is defined
Signed-off-by: Rhea Danzey <rdanzey@element.io>
2023-03-27 15:53:13 -05:00
Devon Hudson 69e3bd82a9
Add dendrite-demo-pinecone cypress tests 2023-03-27 07:57:30 -06:00
Till fa7710315a
Add tests for the Dendrite admin APIs (#3028)
Contains a breaking change, since the endpoints `/_dendrite/admin/evacuateRoom/{roomID}` and `/_dendrite/admin/evacuateUser/{userID}` are now using `POST` instead of `GET`
2023-03-27 15:39:33 +02:00
Till e8b2162a01
Add `/search` tests (#3025) 2023-03-27 11:26:52 +02:00
Till aa1bda4c58
Add AS invite test, fix issue with invitations being processed twice (#3020)
The AS roomserver consumer would receive the events twice, one time as
type `OutputTypeNewInviteEvent` and the other time as
`OutputTypeNewRoomEvent`. 

[skip ci]
2023-03-27 11:26:26 +02:00
Till Faelligen e2d2482ca6
Get the logs for dendrite when installing the chart 2023-03-27 11:07:30 +02:00
Alex Kirk 05f72fc4be
Update docs and sample config for the relay_api (#3011)
This adds an empty `relay_api` section to the sample configuration. For
SQLite environments, or others where a `database.connection_string` is
needed for each section, there should be an entry in the configuration
sample as a basis.

This PR also changes the "Configuring Dendrite" documentation in that
respect.

The requirement was introduced in #2917. When upgrading dendrite, it
will complain about `relay_api.database.connection_string` not being
configured.

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Alex Kirk <akirk@users.noreply.github.com>`
2023-03-27 10:55:36 +02:00
Till 234ed603e6
Move every `db.Prepare` to `sqlutil.Statementlist`, remove trace driver (#3026)
Doesn't buy us much, but makes everything a bit more consistent.

Also removes the SQL trace driver, as it is unused and the output is
hard to read anyway.
2023-03-23 13:52:53 +01:00
Till Faelligen cb18ba0230
Upload covdatafiles for each server 2023-03-22 17:36:33 +01:00
Till Faelligen 14085d30ac
Update workflow to not use commas when joining names 2023-03-22 16:01:12 +01:00
Till Faelligen a4400bdd76
Sytest coverage file 2023-03-22 14:58:36 +01:00
Till Faelligen b741d38e10
Update Workflow 2023-03-22 14:51:18 +01:00
Till Faelligen 6948d16527
Update Go, use go tool covdata for coverage files? 2023-03-22 14:50:21 +01:00
Till 5e85a00cb3
Remove `BaseDendrite` (#3023)
Removes `BaseDendrite` to, hopefully, make testing and composing of
components easier in the future.
2023-03-22 09:21:32 +01:00
Till Faelligen ec6879e5ae
Update GMSL to fix #3013 2023-03-21 16:04:51 +01:00
Till Faelligen 0459d2b9e5
Make "m.upload.size" optional 2023-03-20 09:24:00 +01:00
Till 5579121c6f
Preparations for removing `BaseDendrite` (#3016)
Preparations to actually remove/replace `BaseDendrite`.
Quite a few changes:
- SyncAPI accepts an `fulltext.Indexer` interface (fulltext is removed
from `BaseDendrite`)
- Caches are removed from `BaseDendrite`
- Introduces a `Router` struct (likely to change)
  - also fixes #2903
- Introduces a `sqlutil.ConnectionManager`, which should remove
`base.DatabaseConnection` later on
- probably more
2023-03-17 11:09:45 +00:00
Boris Rybalkin d88f71ab71
simplify unix socket permission format (#3014)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Boris Rybalkin <ribalkin@gmail.com>`
2023-03-16 08:51:21 +01:00
Till Faelligen 2c58bab6a8
Fix UTs on x86 2023-03-15 08:21:00 +01:00
Till 74dc54684b
Version 0.12.0 (#3009) 2023-03-13 20:02:57 +01:00
Till 232aef016c
Add basic runtime tracing (#2996)
This allows us in almost all places to use regions to further trace down
long running tasks.
Also removes an unused function.
2023-03-13 16:45:14 +01:00
Till 689b5ee72f
Change default stats reporting endpoint (#3007)
It's the same instance we report to, only using the subdomain.
2023-03-10 12:27:08 +01:00
Till c7303cbf76
Update dependencies (#3006)
In preparation for a new release, let's also update a few dependencies.
2023-03-10 10:32:50 +01:00
Till 70322699ab
Unset `RoomServerEvent`, since we can't be sure that `Set` actually updates the cached entry (#3002)
This should deflake UTs and be more correct in terms of getting
`Events`.
`Events` tries to fetch the event from the cache first and may get an
unredacted event from it, while it should already be redacted.
2023-03-09 09:52:13 +01:00
Till Faelligen baef523cb0
Fix invalid roomNID returned 2023-03-07 15:35:08 +01:00
Till Faelligen 11a3fcc6cb
RoomServerEvents are mutable, given they can be redacted 2023-03-06 17:58:08 +01:00
Till Faelligen a684b850b9
Actually ignore the error if we were able to backfill events 2023-03-06 17:45:21 +01:00
Till 7d83f8b633
Add tests for `UpdateRelations` (#2999)
This also fixes an issue regarding updates to relations for invalid
events, which could result in us retrying said event over and over
again, if we fail to unmarshal the event to
`gomatrixserverlib.RelationContent`, this was discovered by
`@sleroq:virto.community`
2023-03-06 12:43:59 +01:00
Tim McCormack 7fc839f751
Update admin-promotion instructions; clarify ID for evacuation (#2997)
Table name has changed since instructions were written.

There's probably a better way to describe how to get the internal room
ID than I've attempted here, so feel free to adjust as needed. (It may
even be good to show an example of what an internal room ID looks like,
e.g. `!nc93825:example.com`)

### Pull Request Checklist

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
  * Doc-only change
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)

Signed-off-by: `Tim McCormack <cortex@brainonfire.net>`

---------

Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
2023-03-05 18:42:38 +01:00
Till Faelligen 56b28b01db
Update the cache with the redacted event 2023-03-03 14:49:41 +01:00
Till 9bcd0a2105
Make redaction check easier to read (#2995)
We need to check the redaction PL in Dendrite, if we do it in GMSL, we
end up not sending the event to the output stream because it will be
rejected.

---------

Co-authored-by: kegsay <kegan@matrix.org>
2023-03-03 14:03:17 +01:00
Robin Westerik 7cde99a7a7
Updated instructions and references to monolith to their new names (#2994)
Currently, the documentation makes use of the old names for the binary
and configuration files. This updates the documentation so that users
can follow the guide without issues again.
These changes don't require any go unit tests because it does not modify
any golang code.

Signed-off-by: `Robin Westerik <gh@westerik.me>`
2023-03-03 10:20:53 +01:00
Boris Rybalkin 6b1c9eafa9
unix socket support (#2974)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Boris Rybalkin <ribalkin@gmail.com>`

I need this for Syncloud project (https://github.com/syncloud/platform)
where I run multiple apps behind an nginx on the same RPi like device so
unix socket is very convenient to not have port conflicts between apps.
Also someone opened this Issue:
https://github.com/matrix-org/dendrite/issues/2924

---------

Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
2023-03-01 22:57:30 +01:00
Till 6c20f8f742
Refactor `StoreEvent`, add `MaybeRedactEvent`, create an `EventDatabase` (#2989)
This PR changes the following:
- `StoreEvent` now only stores an event (and possibly prev event),
instead of also doing redactions
- Adds a `MaybeRedactEvent` (pulled out from `StoreEvent`), which should
be called after storing events
- a few other things
2023-03-01 17:06:47 +01:00
Till Faelligen 1aa70b0f56
Fix UTs 2023-03-01 15:09:10 +01:00
Till Faelligen f1ccfcf150
Only run CI if there are changes to go files or the workflow [skip ci] 2023-02-28 15:35:53 +01:00
Till Faelligen 086e205eba
Deploy on gh-pages push 2023-02-28 15:15:19 +01:00
genofire 7fff7cd2ac
feat(helm): add prometheus resources to monitor (#2958)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

---

I do not know, how you run helm-docs ....


otherwise i would like to add somewhere:
````markdown
* Works well with [Prometheus Operator](https://prometheus-operator.dev/) ([Helmchart](https://artifacthub.io/packages/helm/prometheus-community/kube-prometheus-stack)) and there setup of [Grafana](https://grafana.com/grafana/), by enabling following values:
```yaml
prometheus:
  servicemonitor:
    enabled: true
    labels:
      release: "kube-prometheus-stack"
  rules:
    enabled: true # will deploy alert rules
    additionalLabels:
      release: "kube-prometheus-stack"
grafana:
  dashboards:
    enabled: true # will deploy default dashboards 
```
PS: The labels `release=kube-prometheus-stack` is setup with the helmchart of the Prometheus Operator. For Grafana Dashboards it maybe need scan enable to scan in correct namespaces (or ALL), enabled by `sidecar.dashboards.searchNamespace` in [Helmchart of grafana](https://artifacthub.io/packages/helm/grafana/grafana) (which is part of PrometheusOperator, so `grafana.sidecar.dashboards.searchNamespace`) 
````

Maybe also put somewhere the Screenshot of that Grafana Dashboard:
https://grafana.com/grafana/dashboards/13916-dendrite/

---


@S7evinK do you take a look?

Signed-off-by: genofire <geno+dev@fireorbit.de>
2023-02-28 14:18:26 +01:00
Devon Hudson eddf31f915
Fix lint error 2023-02-24 15:49:51 -07:00
Devon Hudson b28406c7d0
Tweaks to pinecone demo to shutdown more cleanly 2023-02-24 15:41:47 -07:00
Till Faelligen 3d31b131fc
Cache all the things 2023-02-24 11:45:01 +01:00
Till ad07b169b8
Refactor `StoreEvent` and create a new `RoomDatabase` interface (#2985)
This PR changes a few things:
- It pulls out the creation of several NIDs from the `StoreEvent`
function to make the functions more reusable
- Uses more caching when using those NIDs to avoid DB round trips
2023-02-24 09:40:20 +01:00
David Schneider e6aa0955ff
Unify logging by using logrus for jetstream logs (#2976)
I guess tests for the logging is rather unusual so I omitted tests for
this change.

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `David Schneider <dsbrng25b@gmail.com>`

---------

Signed-off-by: David Schneider <dsbrng25b@gmail.com>
2023-02-24 08:56:53 +01:00
Yoann N d34277a6c0
Update README sample config link (#2987)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately
2023-02-23 09:07:20 +01:00
Till Faelligen c8ca23acdb
Fix building Element web in CI 2023-02-20 15:28:45 +01:00
Till 7f114cc538
Fix issue where device keys are removed if a device ID is reused (#2982)
Fixes https://github.com/matrix-org/dendrite/issues/2980
2023-02-20 15:26:09 +01:00
Till 4594233f89
Merge keyserver & userapi (#2972)
As discussed yesterday, a first draft of merging the keyserver and the
userapi.
2023-02-20 14:58:03 +01:00
dependabot[bot] bd6f0c14e5
Bump golang.org/x/net from 0.5.0 to 0.7.0 (#2979)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to
0.7.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="8e2b117aee"><code>8e2b117</code></a>
http2/hpack: avoid quadratic complexity in hpack decoding</li>
<li><a
href="547e7edf38"><code>547e7ed</code></a>
http2: avoid referencing ResponseWrite.Write parameter after
returning</li>
<li><a
href="39940adcaa"><code>39940ad</code></a>
html: parse comments per HTML spec</li>
<li><a
href="87ce33ecb4"><code>87ce33e</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="415cb6d518"><code>415cb6d</code></a>
all: fix some comments</li>
<li><a
href="7e3c19ca52"><code>7e3c19c</code></a>
all: correct typos in comments</li>
<li><a
href="296f09aa38"><code>296f09a</code></a>
http2: case insensitive handling for 100-continue</li>
<li><a
href="f8411da775"><code>f8411da</code></a>
nettest: fix tests on dragonfly and js/wasm</li>
<li>See full diff in <a
href="https://github.com/golang/net/compare/v0.5.0...v0.7.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/net&package-manager=go_modules&previous-version=0.5.0&new-version=0.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/matrix-org/dendrite/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-17 19:33:40 +01:00
dependabot[bot] 22c4736495
Bump golang.org/x/image from 0.1.0 to 0.5.0 (#2978)
Bumps [golang.org/x/image](https://github.com/golang/image) from 0.1.0
to 0.5.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="e6c2a4cdd5"><code>e6c2a4c</code></a>
tiff: don't pre-allocate giant slices before reading</li>
<li><a
href="3db422c472"><code>3db422c</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="bb712eb67b"><code>bb712eb</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="9b8a3be425"><code>9b8a3be</code></a>
font/plan9font: limit the font image's dimensions</li>
<li><a
href="c5235ae222"><code>c5235ae</code></a>
font/plan9font: fix byteoffset for non-zero origin</li>
<li><a
href="0888fdd524"><code>0888fdd</code></a>
font/plan9font: fix bounds overflow</li>
<li><a
href="9fdfde75ec"><code>9fdfde7</code></a>
go.mod: update golang.org/x dependencies</li>
<li>See full diff in <a
href="https://github.com/golang/image/compare/v0.1.0...v0.5.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/image&package-manager=go_modules&previous-version=0.1.0&new-version=0.5.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/matrix-org/dendrite/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-17 16:33:49 +01:00
Till f0805071d5
Fix SQLite `session_id` (#2977)
This fixes an issue with device_id/session_ids.
If a `device_id` is reused, we would reuse the same `session_id`, since
we delete one device and insert a new one directly, resulting in the
query to get a new `session_id` to return the previous session_id.
(`SELECT count(access_token)`)
2023-02-17 11:39:46 +01:00