Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proposal] CameraView minimal port #259

Open
1 of 10 tasks
pictos opened this issue Jan 25, 2022 · 58 comments · May be fixed by #1758
Open
1 of 10 tasks

[Proposal] CameraView minimal port #259

pictos opened this issue Jan 25, 2022 · 58 comments · May be fixed by #1758
Assignees
Labels
approved This Proposal has been approved and is ready to be added to the Toolkit champion A member of the .NET MAUI Toolkit core team has chosen to champion this feature in-progress proposal A fully fleshed out proposal describing a new feature in syntactic and semantic detail

Comments

@pictos
Copy link
Member

pictos commented Jan 25, 2022

[CameraView minimal port]

  • Proposed
  • Prototype: Not Started
  • Implementation: Not Started
    • iOS Support
    • Android Support
    • macOS Support
    • Windows Support
  • Unit Tests: Not Started
  • Sample: Not Started
  • Documentation: Not Started

Link to Discussion

Summary

Today on XCT we have a very complete CameraView implementation but with a lot of bugs and a complex codebase to maintain. In the table below you can see all features implemented by each platform.

Android iOS Windows
Preview
Preview Aspect 🚫 🚫
Mirror preview 🚫 🚫
Zoom
Take photo
Flash
Take video
Record audio
Video stabilization

My plan, described in discussion #206, is to drop some features for the v1.0 of this lib. and use CameraX implementation for Android. You can see in the table below what features we will have for v1.0.

Android iOS Windows
Preview
Take photo
Flash

After this first implementation, we can add the missing features to all platforms.

Motivation

The motivation for this plan was the fact that CameraView lacks in quality right now and this's our chance to fix it and provide a more reliable Camera control for our community.

Detailed Design

CameraView.shared.cs

public class CameraView : View
{
  public static readonly BindableProperty IsCameraViewBusyProperty;
  public static readonly BindableProperty IsAvailableProperty;
  public static readonly BindableProperty FlashModeProperty;
  
  // Use WeakEventManager 
  public event EventHandler<bool>? OnAvailable;
  public event EventHandler<MediaCapturedEventArgs>? MediaCaptured; 
  public event EventHandler<string>? MediaCaptureFailed;
  
  public bool IsCameraViewBusy { get; }
  public bool IsAvailable { get; }
  public CameraOptions CameraOptions { get; set; } 
  public void Shutter();
}

Usage Syntax

XAML Usage

<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:xct="http://xamarin.com/schemas/2020/toolkit"
             x:Class="MyLittleApp.MainPage">

     <StackLayout>
        <xct:CameraView
                x:Name="cameraView"
                FlashMode="On"
                MediaCaptured="CameraView_MediaCaptured"
                HorizontalOptions="FillAndExpand"
                VerticalOptions="FillAndExpand" />
    </StackLayout>
</ContentPage>

C# Usage

class MyPage : ContentPage
{
  public MyPage()
  {
    Content = new CameraView
    {
      FlashMode = CameraFlashMode.Off,
    }.FillAndExpand()
     .Invoke(cameraView => cameraView.MediaCaptured += HandleMediaCaptured);
  }

  void HandleMediaCaptured(object? sender, MediaCapturedEventArgs e)
  {
    //...
  }
}

Drawbacks

We will (probably*) break our users on v1.0 since the API will be missing a lot of features.

  • there's a chance that we can implement all features for v1.0 if we have a lot of community help.

Alternatives

  • Follow the [Proposal] CameraView #106 proposal and just port the XCT code to here, but with that, we will bring all issues (if we don't solve them during the port, what I believe it's very hard to do)

Unresolved Questions

@pictos pictos added new proposal A fully fleshed out proposal describing a new feature in syntactic and semantic detail labels Jan 25, 2022
@ghost ghost added this to Proposal Submitted in New Feature Proposals Jan 25, 2022
@VladislavAntonyuk
Copy link
Collaborator

I suggest to add TakeVideo. In that case we cover major Camera features.

@pictos
Copy link
Member Author

pictos commented Jan 25, 2022

@VladislavAntonyuk I would say to let that for the next release, for now. If things move faster, we can bring this feature. Remembering that TakeVideo implies audio record as well, so is one more item to check and test

@brminnick
Copy link
Collaborator

Thanks Pedro! This is a great start! Here's a couple recommendations I have for the API design:

public bool IsBusy { get; set; }

  • Can we rename this to something more descriptive?
    • I.e. What does IsBusy mean? Does it mean the CameraView is in use? Or does it mean the the camera shutter is in use? etc
  • Should this be { get; private set; }? I doubt we'd want the user to toggle IsBusy.

public bool IsAvailable {get; set; }

  • I recommend removing this property, and following the Maui.Essentials example of throwing a FeatureNotSupportedException if a user attempts to use the CameraView on a device where it is not supported
    • If we decide instead to keep this property, I recommend making it's setter private, e.g. { get; private set; }

public void Shutter()

  • Let's change this name to be a verb (an action word), and be more descriptive, e.g. CloseShutter, OpenShutter, etc

CameraOptions

  • Could you include the API Surface for CameraOptions in the Detailed Design?

public event EventHandler<bool>? OnAvailable

  • Could this be changed to public event EventHandler<CameraAvailabilityStatus> CameraAvailablityChanged;?
    • CameraAvailabilityStatus would be an enum
     public enum CameraAvailabilityStatus { Unknown, Available, NotAvailable }
  • Let's use WeakEventManager to ensure we don't induce memory leaks and that the event doesn't need to be nullable
    readonly WeakEventManager eventManager = new();
    
    public event EventHandler<CameraAvailabilityStatus> CameraAvailablityChanged
    {
      add => eventManager.AddEventHandler(value);
      remove => eventManager.RemoveEventHandler(value);
    }

public event EventHandler<MediaCapturedEventArgs>? MediaCaptured

  • Let's use WeakEventManager to ensure we don't induce memory leaks and that the event doesn't need to be nullable
    readonly WeakEventManager eventManager = new();
    
    public event EventHandler<MediaCapturedEventArgs> MediaCaptured
    {
      add => eventManager.AddEventHandler(value);
      remove => eventManager.RemoveEventHandler(value);
    }

public event EventHandler<string>? MediaCaptureFailed

  • Let's use WeakEventManager to ensure we don't induce memory leaks and that the event doesn't need to be nullable
    readonly WeakEventManager eventManager = new();
    
    public event EventHandler<string> MediaCaptureFailed
    {
      add => eventManager.AddEventHandler(value);
      remove => eventManager.RemoveEventHandler(value);
    }

@brminnick brminnick moved this from Proposal Submitted to Proposal Championed in New Feature Proposals Feb 3, 2022
@ghost ghost added champion A member of the .NET MAUI Toolkit core team has chosen to champion this feature and removed new labels Feb 3, 2022
@brminnick brminnick moved this from Proposal Championed to Proposal Approved in New Feature Proposals Feb 3, 2022
@ghost ghost added approved This Proposal has been approved and is ready to be added to the Toolkit help wanted This proposal has been approved and is ready to be implemented labels Feb 3, 2022
@brminnick brminnick mentioned this issue May 2, 2022
10 tasks
@pictos pictos removed the help wanted This proposal has been approved and is ready to be implemented label May 31, 2022
@Mzazvor
Copy link

Mzazvor commented Jun 13, 2022

Hi, Is this camera view also the solution to barcode scanning in MAUI? It seems that way from some Zxing repos comments. Is there any estimate of when this could be avaliable ?

Thanx.

PS: If this question does not belong here im sorry and feel free to delete it.

@pictos
Copy link
Member Author

pictos commented Jun 13, 2022

@Mzazvor, So this cameraView will not be the solution for barcode scanning, the ZXing is another project that you can see here. But there are plans to use the CameraView from this package on the ZXing project.

I can't tell you right now when we will release this feature. It's in progress the Android implementation is almost done

@thra02ad
Copy link

thra02ad commented Aug 7, 2022

Hi all - wanted to ask what’s the status on this? I am not a developer myself but I am building an app where I need a customized camera as part of it. Thanks in advance! :-)

@pictos
Copy link
Member Author

pictos commented Aug 7, 2022

Hey, @thra02ad the implementation for Android is pretty much done... I'm learning about the camera on iOS. I'm building some CameraViews apps in swift to see how that goes and then I'll move to C# and start an implementation for this lib. Right now I don't have any deadline to share with you

@thra02ad
Copy link

thra02ad commented Aug 8, 2022

Hey, @thra02ad the implementation for Android is pretty much done... I'm learning about the camera on iOS. I'm building some CameraViews apps in swift to see how that goes and then I'll move to C# and start an implementation for this lib. Right now I don't have any deadline to share with you

Thank you so much @pictos for swift reply. Apologies for multiple questions, and fully understand that deadline is unknown at the moment. But would you say realistically there is a chance that it will come for iOS this year? Just trying to figure out contingency plans for my app (wait for it vs change back to Xamarin). :-)

@pictos
Copy link
Member Author

pictos commented Aug 8, 2022

@thra02ad my intention is to have this ticket closed before December kkkkk

@thra02ad
Copy link

thra02ad commented Aug 8, 2022

@thra02ad my intention is to have this ticket closed before December kkkkk

Thank you very much for coming back to me! Highly appreciated

@mapo80
Copy link

mapo80 commented Aug 27, 2022

Hi @pictos, can you share github branch where Is your implementation so we can help you to complete this task?

Thanks

@pictos
Copy link
Member Author

pictos commented Aug 27, 2022

@mapo80, here is the branch. Right now I'm blocked by the tooling... I couldn't deploy/debug using a physical device... So I'll wait until the next release and see if that fixes my issue and move forward.

@andreasbrostencab
Copy link

Great news there is a maui version on the way! :)

When using the cameraview from XCT, the flash causes over (or sometimes under) exponation of the image. Often making it way to white, if taking a photo in a dark room. Will photos with flash work better with this MAUI-version?

@pictos
Copy link
Member Author

pictos commented Sep 1, 2022

@andreasbrostencab, the short answer is "I hope so". The long answer is, that we use the default and simple APIs from the Cameras APIs, because we want to be more general and simple... So we will expose APIs to control the flash intensity or something like, but the new implementation will be easier to expand.

@andreasbrostencab
Copy link

Hope work is going well with this new Cameraview. I have another question around it.
In the Xamarin version, you can't zoom to a lower value then 1.0. Now a days meny phones have "wide angel" letting the user "zoom out" to e.g. 0.5. Will this be supported in this new cameraview?

@CliffAgius
Copy link
Contributor

This is awesome @hjam40 and could be a perfect starting point for this proposal.

@bijington
Copy link
Contributor

bijington commented Mar 10, 2023

@hjam40 thank you for sharing this. I am taking over this proposal from @pictos so I will aim to see what he has implemented in this initial PR at: #1080 and then see if we have some kind of overlap in terms of what we want to achieve.

@brminnick brminnick linked a pull request Mar 10, 2023 that will close this issue
@nurfitri97
Copy link

Is it possible to use it in the Blazor Webview?

@bijington
Copy link
Contributor

Is it possible to use it in the Blazor Webview?

No this proposal won't be providing support for Blazor WebView

@brminnick brminnick removed the needs discussion Discuss it on the next Monthly standup label May 4, 2023
@zhitaop
Copy link

zhitaop commented Jul 4, 2023

Hi, any updates for this proposal? We are looking to implement a camera view in our MAUI app and are considering different options

@bijington
Copy link
Contributor

It is on my list of things to pick up, hopefully going to take a look in August.

@StonedHackerman
Copy link

maybe you could yoink some code from this: https://github.com/hjam40/Camera.MAUI

@Dresel
Copy link

Dresel commented Aug 21, 2023

@bijington would you mind if I takeover some parts or start with an initial draft PR?

@bijington
Copy link
Contributor

@bijington would you mind if I takeover some parts or start with an initial draft PR?

Please feel free to get involved. I see you have grabbed @pictos draft PR for this work

@Dresel
Copy link

Dresel commented Aug 30, 2023

@bijington would you mind if I takeover some parts or start with an initial draft PR?

Please feel free to get involved. I see you have grabbed @pictos draft PR for this work

Alright, let's go. I have scheduled sometime next weeks, let's see how far I get.

@pictos
Copy link
Member Author

pictos commented Aug 30, 2023

@Dresel feel free to ask questions during the implementation. The Android part is pretty much done, the next one should be iOS. If you need/prefer real-time conversation join our discord server here

@kfrancis
Copy link

Will the proposal API allow for real-time usage of the camera frame data (ie, constant vs discreet capture) so that we can do things like barcode (by far the most requested use of this feature) or OCR (tesseract)?

@pictos
Copy link
Member Author

pictos commented Aug 30, 2023

@kfrancis you can see what will be supported on the first spec of this issue

@peruchali
Copy link

Hi folks, just curious if there's any update on this?

@Dresel
Copy link

Dresel commented Oct 25, 2023

Work in progress, see #1387 (comment)

@LeoJHarris
Copy link

Jumping on the back of this, thread as we are middle of migrating to MAUI. We want to find the easiest route with least friction on integrating a camera plugin and hoping this will be included very soon in this library so we wont have to find a temporary work around library. Is there any ETA for this to be completed?

@mroseberry99
Copy link

Any updates on the CameraView, specifically for Android and iOS? Is this still being worked on?

@bijington
Copy link
Contributor

I believe @jfversluis is waiting on some feedback on this. And if that doesn't pan out I'll be jumping on this

@jfversluis
Copy link
Member

Correct! I actually pinged them today, which is people from the community who have a working CameraView in their app and are willing to put that in the Toolkit so we have the support for MAUI. Hopefully I'll get word on that soon and else @bijington has offered to take a look, so should be news soon!

@mroseberry99
Copy link

Fantastic guys, can't wait to get it. Thanks for the update.

@jfversluis jfversluis added the needs discussion Discuss it on the next Monthly standup label Mar 7, 2024
@bijington
Copy link
Contributor

Not sure if this should be included in the description but this is the API design that the kind team are going to be contributing:

CameraView

The UI element/view that can be added to an applications UI and renders the preview from a connected camera.

public class CameraView
{
    public static readonly BindableProperty SelectedCameraProperty; // Or possibly ActiveCameraProperty
    public static readonly BindableProperty ZoomFactorProperty;
    public static readonly BindableProperty ResolutionProperty;
    public static readonly BindableProperty IsCameraBusyProperty;
    public static readonly BindableProperty IsAvailableProperty;
    public static readonly BindableProperty FlashModeProperty;
    public static readonly BindableProperty IsTorchOnProperty;

    public event EventHandler<MediaCapturedEventArgs> MediaCaptured;
    public event EventHandler<MediaCaptureFailedEventArgs> MediaCaptureFailed;

    public CameraInfo? SelectedCamera { get; set; } // Or possibly ActiveCamera
    public float ZoomFactor { get; set; }
    public Size Resolution { get; set; } // I believe this only covers the capture resolution - perhaps it should be named `CaptureResolution`?
    public bool IsCameraBusy { get; set; }
    public bool IsAvailable { get; set; }
    public bool IsTorchOn { get; set; }
    public CameraFlashMode FlashMode { get; set; }

    public void Shutter()
    public void Start()
    public void Stop()
}

CameraFlashMode

Enumeration of the possible modes the flash can be set to.

public enum CameraFlashMode
{
    Off,
    On,
    Auto
}

CameraInfo

Provides information about a camera that is connected to a device.

public class CameraInfo
{
    public string Name { get; }

    public string DeviceId { get; }

    public CameraPosition Position { get; }

    public bool IsFlashSupported { get; } // Could/should we enhance this to be a list of capabilities to cover other features like Auto Focus, etc.?

    public float MinimumZoomFactor { get; }

    public float MaximumZoomFactor { get; }

    public ObservableCollection<Size> AvailableResolutions { get; } // two questions; 1. does this need to be observable? 2. maybe SupportedResolutions is a better name?
}

CameraPosition

Enumeration of the possible positions a camera can take.

public enum CameraPosition // Should we have an External value? Can we determine that?
{
    Unknown,
    Rear,
    Front
}

CameraProvider

Provides the ability to obtain information about the cameras connected to the device.

public class CameraProvider
{
    public ObservableCollection<CameraInfo> AvailableCameras { get; }

    public void RefreshAvailableCameras();
}

It would be great if the CameraView could make use of the CameraProvider class internally by default but allow a developer to interact with it as well.

@zhitaop zhitaop linked a pull request Mar 17, 2024 that will close this issue
@aritchie
Copy link

Any chance you guys would consider adding the ability to have a stream come out of the cameraview for processing through things like AI, barcode scanners, etc?

public class CameraView
{
    public Stream OpenStream();
}

Maybe a binding point to Command?

@bijington
Copy link
Contributor

Any chance you guys would consider adding the ability to have a stream come out of the cameraview for processing through things like AI, barcode scanners, etc?

public class CameraView

{

    public Stream OpenStream();

}

Maybe a binding point to Command?

Yes definitely! There isn't anything really documented but this is something that we hope to follow up on once the CameraView gets merged.

The current thinking is to provide potentially 2 mechanisms of achieving this:

  • through some level of access at the platform level, this is to avoid unnecessary allocations/conversions between platform types and common types if a dev wants to process the data on the platform itself.
  • through some common API that is accessible in the common code.

The thought behind this would allow developers to build something as performant as possible which would likely be important if doing something like image processing.

That's a just a conceptual description but how does that sound? I know we have a discussion on barcode scanning that I've been meaning to update. This would likely be a first good use case for it.

@brminnick brminnick removed the needs discussion Discuss it on the next Monthly standup label May 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved This Proposal has been approved and is ready to be added to the Toolkit champion A member of the .NET MAUI Toolkit core team has chosen to champion this feature in-progress proposal A fully fleshed out proposal describing a new feature in syntactic and semantic detail
Projects
New Feature Proposals
Proposal Approved
Development

Successfully merging a pull request may close this issue.