Set image previews correctly according to their focal points (#899)

* Add serialization of the meta-data and focus objects

These objects are added in some attachments. This commit adds data
classes which are able to serialize these (partially) in preparation
for the ability to honour the focal point information in image
previews.

* Implement correctly honouring the focal point meta-data in previews

This commit adds code which ensures that the image previews of media
attachments to toots are correctly cropped to always show the focal
point of the image (if it is specified). It should not in any way
influence how previews of media without a focal point are shown.

To achieve the correct crop on the image a few components were
needed:

First of all we needed a way to influence how the image is cropped
into the ImageView. It turns out that the preferred way to do this is
by setting the ScaleType to MATRIX and adjusting the matrix of the
image as needed. This matrix allows us to scale and transform the
image in the way we need to make sure that the focal point is visible
within the view. For this purpose we have the FocalPointEnforcer which
can calculate and set the appropriate matrix on an ImageView as soon
as the image is loaded.

However a second problem is that we need to make sure that this matrix
is updated whenever the size of the ImageView changes. The size might
change for example because the orientation of the device changed from
portrait to landscape or vice versas, or for a number of other reasons
such as the screen being split vertically or something like that.

To be able to hook onto this event we need to create a new extended
version of the ImageView class, which we call
MediaPreviewImageView. This class behaves exactly the same as a normal
ImageView, however if the focalPointEnforcer of this view is set, then
it will call this enforcer to update the image matrix any time the
size is changed.

So this commit changes all media previews in the item_status.xml and
item_status_detailled.xml layout files to the new
MediaPreviewImageView class. Additionally in the code for loading the
images into the previews a new case is added which tests if there is a
focus attribute in the meta-data. If so it makes sure to create and
set the FocalPointEnforcer.

* Fix typos in documentation comment

"to" -> "too"

* Use static imports to remove clutter in FocalPointEnforcerTest

Instead of duplication Assert. in front of every assertEquals, simply
statically import it.

* Move the MetaData and Focus classes into the Attachment class

Since they are very strongly linked to the attachment class and are
themselves very small.

* Refactor the focal point handling code

- All the code modifying the actual members of the
  MediaPreviewImageView is now in this class itself. This class still
  uses the FocalPointUtil to calculate the new Matrix, but it now
  handles setting this new Matrix itself.

- The FocalPointEnforcer has been renamed to the FocalPointUtil to
  reflect that it only calculates the correct matrix, but doesn't set
  anything on the MediaPreviewImageView.

- The Matrix used to control the cropping of the
  MediaPreviewImageViews is now only allocated a single time per view
  instead of each time the view is resized. This is done by caching
  the Matrix and passing it to the FocalPointUtil to update on each
  resize.

* Only reallocate focalMatrix if it is not yet initialized

This helps prevent unnecessary allocations in the case where
setFocalPoint is called multiple times.

* Change checking of availability of objects to use != null

As pointed out, the 'is' keyword is meant for checking types, not for
checking non-nullness.

* Make updateFocalPointMatrix() return nothing

This makes it clearer that it actually mutates the matrix it is
given.

* Fix bug with transitions crashing the PhotoView

Due to the android transitions for some reason copying the scaletype
from the MediaPreviewImageView to the PhotoView during the transition,
the PhotoView would crash on pictures with a focal point, since
PhotoView doesn't support ScaleType.MATRIX.

This is solved by the workaround of overriding both the getScaleType
and setScaleType methods to ensure that we use the MATRIX type in the
preview and the center_crop type in the PhotoView.

Additionally this commit also makes sure to remove the focal point
when the MediaPreviewImageView is recycled.

* Fix bug in overriden getScaleType

Instead of simply returning the scaleType we need to return the
super.getScaleType() method, to avoid crashing.

* Merge changes from master

Mainly the migration to androidx.
main
jchmrt 6 years ago committed by Konrad Pozniak
parent 721c1f273f
commit 5bdee9329a
  1. 32
      app/src/main/java/com/keylesspalace/tusky/adapter/StatusBaseViewHolder.java
  2. 21
      app/src/main/java/com/keylesspalace/tusky/entity/Attachment.kt
  3. 159
      app/src/main/java/com/keylesspalace/tusky/util/FocalPointUtil.kt
  4. 124
      app/src/main/java/com/keylesspalace/tusky/view/MediaPreviewImageView.kt
  5. 8
      app/src/main/res/layout/item_status.xml
  6. 8
      app/src/main/res/layout/item_status_detailed.xml
  7. 110
      app/src/test/java/com/keylesspalace/tusky/FocalPointUtilTest.kt

@ -19,6 +19,8 @@ import android.widget.ToggleButton;
import com.keylesspalace.tusky.R;
import com.keylesspalace.tusky.entity.Attachment;
import com.keylesspalace.tusky.entity.Attachment.Focus;
import com.keylesspalace.tusky.entity.Attachment.MetaData;
import com.keylesspalace.tusky.entity.Emoji;
import com.keylesspalace.tusky.entity.Status;
import com.keylesspalace.tusky.interfaces.StatusActionListener;
@ -28,6 +30,7 @@ import com.keylesspalace.tusky.util.HtmlUtils;
import com.keylesspalace.tusky.util.LinkHelper;
import com.keylesspalace.tusky.util.SmartLengthInputFilter;
import com.keylesspalace.tusky.util.ThemeUtils;
import com.keylesspalace.tusky.view.MediaPreviewImageView;
import com.keylesspalace.tusky.viewdata.StatusViewData;
import com.mikepenz.iconics.utils.Utils;
import com.squareup.picasso.Picasso;
@ -53,7 +56,7 @@ abstract class StatusBaseViewHolder extends RecyclerView.ViewHolder {
private ImageButton moreButton;
private boolean favourited;
private boolean reblogged;
private ImageView[] mediaPreviews;
private MediaPreviewImageView[] mediaPreviews;
private ImageView[] mediaOverlays;
private TextView sensitiveMediaWarning;
private View sensitiveMediaShow;
@ -83,7 +86,7 @@ abstract class StatusBaseViewHolder extends RecyclerView.ViewHolder {
moreButton = itemView.findViewById(R.id.status_more);
reblogged = false;
favourited = false;
mediaPreviews = new ImageView[]{
mediaPreviews = new MediaPreviewImageView[] {
itemView.findViewById(R.id.status_media_preview_0),
itemView.findViewById(R.id.status_media_preview_1),
itemView.findViewById(R.id.status_media_preview_2),
@ -155,7 +158,6 @@ abstract class StatusBaseViewHolder extends RecyclerView.ViewHolder {
LinkHelper.setClickableText(this.content, emojifiedText, mentions, listener);
} else {
LinkHelper.setClickableMentions(this.content, mentions, listener);
}
if(TextUtils.isEmpty(this.content.getText())) {
this.content.setVisibility(View.GONE);
@ -300,10 +302,26 @@ abstract class StatusBaseViewHolder extends RecyclerView.ViewHolder {
.load(mediaPreviewUnloadedId)
.into(mediaPreviews[i]);
} else {
Picasso.with(context)
.load(previewUrl)
.placeholder(mediaPreviewUnloadedId)
.into(mediaPreviews[i]);
MetaData meta = attachments.get(i).getMeta();
Focus focus = meta != null ? meta.getFocus() : null;
if (focus != null) { // If there is a focal point for this attachment:
mediaPreviews[i].setFocalPoint(focus);
Picasso.with(context)
.load(previewUrl)
.placeholder(mediaPreviewUnloadedId)
// Also pass the mediaPreview as a callback to ensure it is called
// initially when the image gets loaded:
.into(mediaPreviews[i], mediaPreviews[i]);
} else {
mediaPreviews[i].removeFocalPoint();
Picasso.with(context)
.load(previewUrl)
.placeholder(mediaPreviewUnloadedId)
.into(mediaPreviews[i]);
}
}
final Attachment.Type type = attachments.get(i).getType();

@ -30,6 +30,7 @@ data class Attachment(
var url: String,
@SerializedName("preview_url") val previewUrl: String,
@SerializedName("text_url") val textUrl: String?,
val meta: MetaData?,
var type: Type,
var description: String?
) : Parcelable {
@ -57,4 +58,24 @@ data class Attachment(
}
}
}
/**
* The meta data of an [Attachment].
*/
@Parcelize
data class MetaData (
val focus: Focus?
) : Parcelable
/**
* The Focus entity, used to specify the focal point of an image.
*
* See here for more details what the x and y mean:
* https://github.com/jonom/jquery-focuspoint#1-calculate-your-images-focus-point
*/
@Parcelize
data class Focus (
val x: Float,
val y: Float
) : Parcelable
}

@ -0,0 +1,159 @@
/* Copyright 2018 Jochem Raat <jchmrt@riseup.net>
*
* This file is a part of Tusky.
*
* This program is free software; you can redistribute it and/or modify it under the terms of the
* GNU General Public License as published by the Free Software Foundation; either version 3 of the
* License, or (at your option) any later version.
*
* Tusky is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even
* the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
* Public License for more details.
*
* You should have received a copy of the GNU General Public License along with Tusky; if not,
* see <http://www.gnu.org/licenses>. */
package com.keylesspalace.tusky.util
import android.graphics.Matrix
import android.widget.ImageView
import com.keylesspalace.tusky.entity.Attachment.Focus
import com.squareup.picasso.Callback
/**
* Calculates the image matrix needed to maintain the correct cropping for image views based on
* their focal point.
*
* The purpose of this class is to make sure that the focal point information on media
* attachments are honoured. This class uses the custom matrix option of android ImageView's to
* customize how the image is cropped into the view.
*
* See the explanation of focal points here:
* https://github.com/jonom/jquery-focuspoint#1-calculate-your-images-focus-point
*/
object FocalPointUtil {
/**
* Update the given matrix for the given parameters.
*
* How it works is using the following steps:
* - First we determine if the image is too wide or too tall for the view size. If it is
* too wide, we need to crop it horizontally and scale the height to fit the view
* exactly. If it is too tall we need to crop vertically and scale the width to fit the
* view exactly.
* - Then we determine what translation is needed to get the focal point in view. We
* prefer to get the focal point at the center of the preview. However if that would
* result in some part of the preview being empty, we instead align the image so that it
* fills the view, but still the focal point is always in view.
*
* @param viewWidth The width of the imageView.
* @param viewHeight The height of the imageView
* @param imageWidth The width of the actual image
* @param imageHeight The height of the actual image
* @param focus The focal point to focus
* @param mat The matrix to update, this matrix is reset() and then updated with the new
* configuration. We reuse the old matrix to prevent unnecessary allocations.
*
* @return The matrix which correctly crops the image
*/
fun updateFocalPointMatrix(viewWidth: Float,
viewHeight: Float,
imageWidth: Float,
imageHeight: Float,
focus: Focus,
mat: Matrix) {
// Reset the cached matrix:
mat.reset()
// calculate scaling:
val scale = calculateScaling(viewWidth, viewHeight, imageWidth, imageHeight)
mat.preScale(scale, scale)
// calculate offsets:
var top = 0f
var left = 0f
if (isVerticalCrop(viewWidth, viewHeight, imageWidth, imageHeight)) {
top = focalOffset(viewHeight, imageHeight, scale, focalYToCoordinate(focus.y))
} else { // horizontal crop
left = focalOffset(viewWidth, imageWidth, scale, focalXToCoordinate(focus.x))
}
mat.postTranslate(left, top)
}
/**
* Calculate the scaling of the image needed to make it fill the screen.
*
* The scaling used depends on if we need a vertical of horizontal crop.
*/
fun calculateScaling(viewWidth: Float, viewHeight: Float,
imageWidth: Float, imageHeight: Float): Float {
if (isVerticalCrop(viewWidth, viewHeight, imageWidth, imageHeight)) {
return viewWidth / imageWidth
} else { // horizontal crop:
return viewHeight / imageHeight
}
}
/**
* Return true if we need a vertical crop, false for a horizontal crop.
*/
fun isVerticalCrop(viewWidth: Float, viewHeight: Float,
imageWidth: Float, imageHeight: Float): Boolean {
val viewRatio = viewWidth / viewHeight
val imageRatio = imageWidth / imageHeight
return viewRatio > imageRatio
}
/**
* Transform the focal x component to the corresponding coordinate on the image.
*
* This means that we go from a representation where the left side of the image is -1 and
* the right side +1, to a representation with the left side being 0 and the right side
* being +1.
*/
fun focalXToCoordinate(x: Float): Float {
return (x + 1) / 2
}
/**
* Transform the focal y component to the corresponding coordinate on the image.
*
* This means that we go from a representation where the bottom side of the image is -1 and
* the top side +1, to a representation with the top side being 0 and the bottom side
* being +1.
*/
fun focalYToCoordinate(y: Float): Float {
return (-y + 1) / 2
}
/**
* Calculate the relative offset needed to focus on the focal point in one direction.
*
* This method works for both the vertical and horizontal crops. It simply calculates
* what offset to take based on the proportions between the scaled image and the view
* available. It also makes sure to always fill the bounds of the view completely with
* the image. So it won't put the very edge of the image in center, because that would
* leave part of the view empty.
*/
fun focalOffset(view: Float, image: Float,
scale: Float, focal: Float): Float {
// The fraction of the image that will be in view:
val inView = view / (scale * image)
var offset = 0f
// These values indicate the maximum and minimum focal parameter possible while still
// keeping the entire view filled with the image:
val maxFocal = 1 - inView / 2
val minFocal = inView / 2
if (focal > maxFocal) {
offset = -((2 - inView) / 2) * image * scale + view * 0.5f
} else if (focal > minFocal) {
offset = -focal * image * scale + view * 0.5f
}
return offset
}
}

@ -0,0 +1,124 @@
/* Copyright 2018 Jochem Raat <jchmrt@riseup.net>
*
* This file is a part of Tusky.
*
* This program is free software; you can redistribute it and/or modify it under the terms of the
* GNU General Public License as published by the Free Software Foundation; either version 3 of the
* License, or (at your option) any later version.
*
* Tusky is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even
* the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
* Public License for more details.
*
* You should have received a copy of the GNU General Public License along with Tusky; if not,
* see <http://www.gnu.org/licenses>. */
package com.keylesspalace.tusky.view
import android.content.Context
import android.graphics.Matrix
import android.util.AttributeSet
import androidx.appcompat.widget.AppCompatImageView
import com.keylesspalace.tusky.entity.Attachment
import com.keylesspalace.tusky.util.FocalPointUtil
import com.squareup.picasso.Callback
/**
* This is an extension of the standard android ImageView, which makes sure to update the custom
* matrix when its size changes if a focal point is set.
*
* If a focal point is set on this view, it will use the FocalPointUtil to update the image
* matrix each time the size of the view is changed. This is needed to ensure that the correct
* cropping is maintained.
*
* However if there is no focal point set (e.g. it is null), then this view should simply
* act exactly the same as an ordinary android ImageView.
*/
class MediaPreviewImageView
@JvmOverloads constructor(
context: Context,
attrs: AttributeSet? = null,
defStyleAttr: Int = 0
) : AppCompatImageView(context, attrs, defStyleAttr), Callback {
private var focus: Attachment.Focus? = null
private var focalMatrix: Matrix? = null
/**
* Set the focal point for this view.
*/
fun setFocalPoint(focus: Attachment.Focus) {
this.focus = focus
super.setScaleType(ScaleType.MATRIX)
if (focalMatrix == null) {
focalMatrix = Matrix()
}
}
/**
* Remove the focal point from this view (if there was one).
*/
fun removeFocalPoint() {
super.setScaleType(ScaleType.CENTER_CROP)
focus = null
}
/**
* Overridden getScaleType method which returns CENTER_CROP if we have a focal point set.
*
* This is necessary because the Android transitions framework tries to copy the scale type
* from this view to the PhotoView when animating between this view and the detailled view of
* the image. Since the PhotoView does not support a MATRIX scale type, the app would crash
* if we simply passed that on, so instead we pretend that CENTER_CROP is still used here
* even if we have a focus point set.
*/
override fun getScaleType(): ScaleType {
if (focus != null) {
return ScaleType.CENTER_CROP
} else {
return super.getScaleType()
}
}
/**
* Overridden setScaleType method which only accepts the new type if we don't have a focal
* point set.
*
*
*/
override fun setScaleType(type: ScaleType) {
if (focus != null) {
super.setScaleType(ScaleType.MATRIX)
} else {
super.setScaleType(type)
}
}
/**
* Called when the image is first succesfully loaded by Picasso, this function makes sure
* that the custom matrix of this image is initialized if a focus point is set.
*/
override fun onSuccess() {
onSizeChanged(width, height, width, height);
}
// We do not handle the error here, instead it will be handled higher up the call chain.
override fun onError() {
}
/**
* Called when the size of the view changes, it calls the FocalPointUtil to update the
* matrix if we have a set focal point. It then reassigns the matrix to this imageView.
*/
override fun onSizeChanged(width: Int, height: Int, oldWidth: Int, oldHeight: Int) {
if (drawable != null && focus != null && focalMatrix != null) {
scaleType = ScaleType.MATRIX
FocalPointUtil.updateFocalPointMatrix(width.toFloat(), height.toFloat(),
drawable.intrinsicWidth.toFloat(), drawable.intrinsicHeight.toFloat(),
focus as Attachment.Focus, focalMatrix as Matrix)
imageMatrix = focalMatrix
}
super.onSizeChanged(width, height, oldWidth, oldHeight)
}
}

@ -164,7 +164,7 @@
android:layout_marginTop="@dimen/status_media_preview_margin_top"
android:layout_toEndOf="@+id/status_avatar">
<ImageView
<com.keylesspalace.tusky.view.MediaPreviewImageView
android:id="@+id/status_media_preview_0"
android:layout_width="0dp"
android:layout_height="@dimen/status_media_preview_height"
@ -174,7 +174,7 @@
app:layout_constraintTop_toTopOf="parent"
tools:ignore="ContentDescription" />
<ImageView
<com.keylesspalace.tusky.view.MediaPreviewImageView
android:id="@+id/status_media_preview_1"
android:layout_width="0dp"
android:layout_height="@dimen/status_media_preview_height"
@ -186,7 +186,7 @@
tools:ignore="ContentDescription" />
<ImageView
<com.keylesspalace.tusky.view.MediaPreviewImageView
android:id="@+id/status_media_preview_2"
android:layout_width="0dp"
android:layout_height="@dimen/status_media_preview_height"
@ -197,7 +197,7 @@
app:layout_constraintTop_toBottomOf="@+id/status_media_preview_0"
tools:ignore="ContentDescription" />
<ImageView
<com.keylesspalace.tusky.view.MediaPreviewImageView
android:id="@+id/status_media_preview_3"
android:layout_width="0dp"
android:layout_height="@dimen/status_media_preview_height"

@ -167,7 +167,7 @@
android:layout_marginTop="@dimen/status_media_preview_margin_top"
android:layout_marginBottom="4dp">
<ImageView
<com.keylesspalace.tusky.view.MediaPreviewImageView
android:id="@+id/status_media_preview_0"
android:layout_width="0dp"
android:layout_height="@dimen/status_media_preview_height"
@ -177,7 +177,7 @@
app:layout_constraintTop_toTopOf="parent"
tools:ignore="ContentDescription" />
<ImageView
<com.keylesspalace.tusky.view.MediaPreviewImageView
android:id="@+id/status_media_preview_1"
android:layout_width="0dp"
android:layout_height="@dimen/status_media_preview_height"
@ -188,7 +188,7 @@
app:layout_constraintTop_toTopOf="parent"
tools:ignore="ContentDescription" />
<ImageView
<com.keylesspalace.tusky.view.MediaPreviewImageView
android:id="@+id/status_media_preview_2"
android:layout_width="0dp"
android:layout_height="@dimen/status_media_preview_height"
@ -199,7 +199,7 @@
app:layout_constraintTop_toBottomOf="@+id/status_media_preview_0"
tools:ignore="ContentDescription" />
<ImageView
<com.keylesspalace.tusky.view.MediaPreviewImageView
android:id="@+id/status_media_preview_3"
android:layout_width="0dp"
android:layout_height="@dimen/status_media_preview_height"

@ -0,0 +1,110 @@
/* Copyright 2018 Jochem Raat <jchmrt@riseup.net>
*
* This file is a part of Tusky.
*
* This program is free software; you can redistribute it and/or modify it under the terms of the
* GNU General Public License as published by the Free Software Foundation; either version 3 of the
* License, or (at your option) any later version.
*
* Tusky is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even
* the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
* Public License for more details.
*
* You should have received a copy of the GNU General Public License along with Tusky; if not,
* see <http://www.gnu.org/licenses>. */
package com.keylesspalace.tusky
import com.keylesspalace.tusky.util.FocalPointUtil
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Assert.assertFalse
import org.junit.Test
class FocalPointUtilTest {
private val eps = 0.01f
// focal[X|Y]ToCoordinate tests
@Test
fun positiveFocalXToCoordinateTest() {
assertEquals(FocalPointUtil.focalXToCoordinate(0.4f), 0.7f, eps)
}
@Test
fun negativeFocalXToCoordinateTest() {
assertEquals(FocalPointUtil.focalXToCoordinate(-0.8f), 0.1f, eps)
}
@Test
fun positiveFocalYToCoordinateTest() {
assertEquals(FocalPointUtil.focalYToCoordinate(-0.2f), 0.6f, eps)
}
@Test
fun negativeFocalYToCoordinateTest() {
assertEquals(FocalPointUtil.focalYToCoordinate(0.0f), 0.5f, eps)
}
// isVerticalCrop tests
@Test
fun isVerticalCropTest() {
assertTrue(FocalPointUtil.isVerticalCrop(2f, 1f,
1f, 2f))
}
@Test
fun isHorizontalCropTest() {
assertFalse(FocalPointUtil.isVerticalCrop(1f, 2f,
2f,1f))
}
@Test
fun isPerfectFitTest() { // Doesn't matter what it returns, just check it doesn't crash
FocalPointUtil.isVerticalCrop(3f, 1f,
6f, 2f)
}
// calculateScaling tests
@Test
fun perfectFitScaleDownTest() {
assertEquals(FocalPointUtil.calculateScaling(2f, 5f,
5f, 12.5f), 0.4f, eps)
}
@Test
fun perfectFitScaleUpTest() {
assertEquals(FocalPointUtil.calculateScaling(2f, 5f,
1f, 2.5f), 2f, eps)
}
@Test
fun verticalCropScaleUpTest() {
assertEquals(FocalPointUtil.calculateScaling(2f, 1f,
1f, 2f), 2f, eps)
}
@Test
fun verticalCropScaleDownTest() {
assertEquals(FocalPointUtil.calculateScaling(4f, 3f,
8f, 24f), 0.5f, eps)
}
@Test
fun horizontalCropScaleUpTest() {
assertEquals(FocalPointUtil.calculateScaling(1f, 2f,
2f, 1f), 2f, eps)
}
@Test
fun horizontalCropScaleDownTest() {
assertEquals(FocalPointUtil.calculateScaling(3f, 4f,
24f, 8f), 0.5f, eps)
}
// focalOffset tests
@Test
fun toLowFocalOffsetTest() {
assertEquals(FocalPointUtil.focalOffset(2f, 8f, 1f, 0.05f),
0f, eps)
}
@Test
fun toHighFocalOffsetTest() {
assertEquals(FocalPointUtil.focalOffset(2f, 4f, 2f,0.95f),
-6f, eps)
}
@Test
fun possibleFocalOffsetTest() {
assertEquals(FocalPointUtil.focalOffset(2f, 4f, 2f,0.7f),
-4.6f, eps)
}
}
Loading…
Cancel
Save