.NET, एज़्योर फ़ंक्शंस और संज्ञानात्मक सेवाओं के जादू के साथ एक इंटरैक्टिव प्रदर्शन बनाएँ

जनवरी की शुरुआत में, मास्को इलेक्ट्रोम्यूजियम ने ओपन म्यूजियम 2020 प्रदर्शनी में भाग लेने के लिए प्रदर्शनों के एक खुले चयन की घोषणा की । इस लेख में मैं आपको बताऊंगा कि कैसे मैंने एक संज्ञानात्मक चित्र के विचार को बदल दिया, जिसके बारे में मैंने पहले ही लिखा था , एक इंटरैक्टिव प्रदर्शनी में, और कैसे संग्रहालय के छंटने के बाद, यह प्रदर्शनी आभासी हो गई। कटौती के तहत - UWP अनुप्रयोगों में एज़्योर फ़ंक्शंस , बॉट फ्रेमवर्क , संज्ञानात्मक सेवाओं और चेहरे का पता लगाने के लिए एक भ्रमण ।

AI April. , Microsoft AI . — . .

, :

, 2019, People Blending	, 2020, Cognitive Portrait

OpenMuseum, , - . , "" .

, . , "" , , :

• , .
• , , .
• , , .
• , , , "" .

, :

• UWP-, , - . UWP- Raspberry Pi Windows IoT Core. :
  
  • , (face detection).
  • — .
  • , .
• , :
  
  • .
  • , , .
  • Azure Storage.
  • , , Azure Storage .

, , - . , - , , -.

@PeopleBlenderBot. REST API, . , , " ", . , , , " ".

@PeopleBlenderBot , ( ) , , . , . , .

.

UWP

, Windows — " ". , Raspberry Pi Windows 10 IoT Core. , Raspberry Windows , Intel NUC.

:

XAML ( ):

<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
    <Grid.ColumnDefinitions>
        <ColumnDefinition Width="*"/>
        <ColumnDefinition Width="*"/>
    </Grid.ColumnDefinitions>
    <Grid.RowDefinitions>
        <RowDefinition Height="*"/>
        <RowDefinition Height="170"/>
    </Grid.RowDefinitions>
    <Grid x:Name="FacesCanvas" Grid.Row="0" Grid.Column="0">
        <CaptureElement x:Name="ViewFinder" />
        <Rectangle x:Name="FaceRect"/>
        <TextBlock x:Name="Counter" FontSize="60"/>
    </Grid>
    <Grid x:Name="ResultCanvas" Grid.Row="0" Grid.Column="1">
        <Image x:Name="ResultImage" Source="Assets/bgates.jpg"/>
    </Grid>
    <ItemsControl x:Name="FacesLine" Grid.Row="1" Grid.ColumnSpan="2" 
              ItemsSource="{x:Bind Faces,Mode=OneWay}"/>
</Grid>

:

• ViewFinder .
• FaceRect Counter ViewFinder, 3-2-1, .
• ResultImage — , .
• FacesLine , . Faces, Observable. , , Faces.
• XAML .

UWP , . , ( ).

, , , ViewFinder:

MC = new MediaCapture();
var cameras = await DeviceInformation.FindAllAsync(
                              DeviceClass.VideoCapture);
var camera = cameras.First();
var settings = new MediaCaptureInitializationSettings() 
                          { VideoDeviceId = camera.Id };
await MC.InitializeAsync(settings);
ViewFinder.Source = MC;

. FaceDetectionEffect, FaceDetectedEvent :

var def = new FaceDetectionEffectDefinition();
def.SynchronousDetectionEnabled = false;
def.DetectionMode = FaceDetectionMode.HighPerformance;
FaceDetector = (FaceDetectionEffect)
     (await MC.AddVideoEffectAsync(def, MediaStreamType.VideoPreview));
FaceDetector.FaceDetected += FaceDetectedEvent;
FaceDetector.DesiredDetectionInterval = TimeSpan.FromMilliseconds(100);
FaceDetector.Enabled = true;
await MC.StartPreviewAsync();

, FaceDetectedEvent, , . Counter, 3, 2 1. 0, MemoryStream, CallCognitiveFunction, :

var ms = new MemoryStream();
await MC.CapturePhotoToStreamAsync(
      ImageEncodingProperties.CreateJpeg(), ms.AsRandomAccessStream());
var cb = await GetCroppedBitmapAsync(ms,DFace.FaceBox);
Faces.Add(cb);
var url = await CallCognitiveFunction(ms);
ResultImage.Source = new BitmapImage(new Uri(url));

, - REST POST, , - , URL . , URL Source ResultImage, ( UWP ).

. GetCroppedBitmapAsync, , Faces, ( DataBinding).

CallCognitiveFunction , , REST HttpClient:

private async Task<string> CallCognitiveFunction(MemoryStream ms)
{
    ms.Position = 0;
    var resp = await http.PostAsync(function_url,new StreamContent(ms));
    return await resp.Content.ReadAsStringAsync();
}

Azure Function

Azure Functions Python. Azure Functions :

• , . Azure Functions (serverless).
• functions consumption plan, , . (15 ), . , .
• , .
• Azure Function . REST-, , , .
• Azure Function , . , , .

, , . Azure Function, -, . , , — . boilerplate code!

Azure Functions , , - , -, IoT- ..

Python OpenCV — . , Azure Functions Python, — (V2) .

. , . VS Code, — .

CLI:

func init coportrait –python
cd coportrait
func new --name pdraw --template "HTTP trigger"

, Azure Functions Core Tools.

:

• Python ( , __init__.py).
• function.json, , .. , .

HTTP-, function.json :

{
  "scriptFile": "__init__.py",
  "bindings": [
    {
      "authLevel": "function",
      "type": "httpTrigger",
      "direction": "in",
      "name": "req",
      "methods": [ "post" ]},
    {
      "type": "http",
      "direction": "out",
      "name": "$return" 
}]}

, req HTTP-, — HTTP response, ($return). , POST-. "get", .

__init__.py, ( ):

def main(req:func.HttpRequest) -> func.HttpResponse:
    logging.info('Execution begins…')
    return func.HttpResponse(f"Hello {name}!")

, req — . , JPEG, get_body, numpy- OpenCV:

body = req.get_body()
nparr = np.fromstring(body, np.uint8)
img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)

azure.storage.blob.BlockBlobService. , , , , . Azure Python .

cin:

blob = BlockBlobService(account_name=..., account_key=...) 
sec_p = int((end_date-datetime.datetime.now()).total_seconds())
name =  f"{sec_p:09d}-{time.strftime('%Y%m%d-%H%M%S')}"
blob.create_blob_from_bytes("cin",name,body)

. 10 , , . , , . 1 2021 , ( ) , YYYYMMDD-HHMMSS. 2021 ...

Face API , :

cogface = cf.FaceClient(cognitive_endpoint,
                        CognitiveServicesCredentials(cognitive_key))
res = cogface.face.detect_with_stream(io.BytesIO(body),
                        return_face_landmarks=True)
if res is not None and len(res)>0:
    tr = affine_transform(img,res[0].face_landmarks.as_dict())
    body = cv2.imencode('.jpg',tr)[1]
    blob.create_blob_from_bytes("cmapped",name,body.tobytes())

affine_transform . , , JPEG- cmapped.

, , 10 cmapped . list_blobs , 10 islice, imdecode numpy- :

imgs = [ imdecode(blob.get_blob_to_bytes("cmapped",x.name).content) 
         for x in itertools.islice(blob.list_blobs("cmapped"),10) ]
imgs = np.array(imgs).astype(np.float)

, . , float, — np.uint8:

res = (np.average(imgs,axis=0)).astype(np.uint8)

, out, cv2.imencode, create_blob_from_bytes:

b = cv2.imencode('.jpg',res)[1]
r = blob.create_blob_from_bytes("out",f"{name}.jpg",b.tobytes())
result_url = f"https://{act}.blob.core.windows.net/out/{name}.jpg"
return func.HttpResponse(result_url)

— out, , , .

__init__.py, requirements.txt:

azure-functions
opencv-python
azure-cognitiveservices-vision-face
azure-storage-blob==1.5.0

, :

func start

-, URL , Postman. POST, - .
URL function_url UWP-, .

Azure Function , Python Azure Function Azure Portal, Azure CLI:

az functionapp create --resource-group PeopleBlenderBot
    --os-type Linux   --consumption-plan-location westeurope
    --runtime python  --runtime-version 3.7
    --functions-version 2
    --name coportrait --storage-account coportraitstore

Azure Portal , ( ) . Azure CLI , .

, , :

func azure functionapp publish coportrait

Azure Portal, URL:

URL : https://coportrait.azurewebsites.net/api/pdraw?code=geE..e3P==, .. . ( ) function_url UWP-, , !

-

, . — Microsoft Bot Framework -.

, , . , -.

- C# , . - Python, .NET , .

, VS Bot Template Visual Studio, Echo:

PeopleBlenderBot.

Bots\EchoBot.cs, OnMessageActivityAsync. .. turnContext. — Activity, Text, Attachments. , :

if (turnContext.Activity.Attachments?.Count>0)
{
   // do the magic
}
else
{
    await turnContext.SendActivityAsync("Please send picture");
}

, ContentUrl Attachment. if-, http-:

  var http = new HttpClient();
  var resp = await http.GetAsync(Attachments[0].ContentUrl);
  var str = await resp.Content.ReadAsStreamAsync();

Azure Function, POST-. UWP-:

  resp = await http.PostAsync(function_url, new StreamContent(str));
  var url = await resp.Content.ReadAsStringAsync();

URL , . , Hero-card, attachment-:

  var msg = MessageFactory.Attachment(
      (new HeroCard()
      { Images = new CardImage[] { new CardImage(url) } }
      ).ToAttachment());
  await turnContext.SendActivityAsync(msg);

, Bot Framework Emulator:

, , , Azure CLI ARM-. , Azure Portal:

1. Web App Bot , Echo Bot . :
   
   • Bot Connector App, , , Telegram
   • Bot Web App, -,
2. appsettings.json PeopleBlenderBot. App Id App Password, bot connector.
3. Visual Studio PeopleBlenderBot Publish. Existing Web App, , 1. .

, Bot Connector App Azure Portal, , Web Chat:

— ! , Telegram Channels :

, . , Mechanium, Moscow Maker Faire 2019. , - . — Science Art, — — . , , ! - — pull request ! , , . — , - !

-, , Telegram @peopleblenderbot, . ! , , , , . , , .

! , , !