Get best of the week

Gateway Python na InterSystems IRIS

Este artigo se concentra no Python Gateway, um projeto de comunidade de código aberto para a plataforma de dados InterSystems IRIS. Este projeto permite orquestrar qualquer algoritmo de aprendizado de máquina criado em Python (o principal ambiente para muitos cientistas de dados), usar inúmeras bibliotecas prontas para criar rapidamente soluções de AI / ML analíticas, adaptáveis ​​e robóticas, baseadas na plataforma InterSystems IRIS. Neste artigo, mostrarei como o InterSystems IRIS pode orquestrar processos em Python, implementar eficientemente a transferência de dados bidirecional e criar processos de negócios inteligentes.


Plano


  1. Introdução
  2. Ferramentas.
  3. Instalação
  4. API
  5. Interoperabilidade
  6. Caderno Jupyter.
  7. Constatações.
  8. Referências
  9. MLToolkit.

Introdução


Python — , . Python InterSystems IRIS, Python .


(ML) — , , .


. , , . ?


, , , .


— GUI- AI/ML-, , , , BI ( GUI- ), AI/ML. .


, , , . , .


, . , InterSystems IRIS, .


AI/ML- , -. , . , , , .


? . , . , ? . , . , . — . , ? . , , - , .


ML Toolkit — , : , -. Python Gateway ML Toolkit Python ( R Gateway, ML Toolkit R).



, Python, .



  • Python — , . , ML- AI-. ObjectScript, - , , . . . , .
  • Python 3.6.7 64 bit.
  • IDE: PyCharm, . Atelier, Eclipse- Python. VS Code, Python.
  • Notebook: IDE -. — Jupyter.


() :



AI/ML . , . AI/ML- , .



Python Gateway:



    • Windows
    • Linux
    • Mac
  • Docker
    • DockerHub

, . — . , . 0.8, . / , .



Python Gateway , ( ) Python. :


  1. Python 3.6.7 64 bit. Python .
  2. dill: pip install dill.
  3. ObjectScript (.. do $system.OBJ.ImportDir("C:\InterSystems\Repos\Python\isc\py\", "*.cls", "c",,1)) . , , , : write ##class(%EnsembleMgr).EnableNamespace($Namespace, 1).
  4. callout DLL/SO/DYLIB bin InterSystems IRIS. , write ##class(isc.py.Callout).GetLib().

Windows


  1. , PYTHONHOME Python 3.6.7.
  2. , PATH PYTHONHOME ( , ).

Linux (Debian/Ubuntu)


  1. , PATH /usr/lib /usr/lib/x86_64-linux-gnu. /etc/environment .
  2. undefined symbol: _Py_TrueStruct PythonLib. Readme Troubleshooting.

Mac


  1. 3.6.7 Python.org. PATH.

, InterSystems.


Docker


:



Docker InterSystems.


Python Gateway 2019.4.



: docker run -d -p 52773:52773 --name irispy intersystemscommunity/irispy-community:latest, Python Gateway InterSystems IRIS Community Edition. .



- : docker build --force-rm --tag intersystemscommunity/irispy:latest ..
store/intersystems/iris-community:2019.4.0.383.0, , IMAGE.
InterSystems IRIS : `docker build --build-arg IMAGE=store/intersystems/iris:2019.4.0.383.0 --force-rm --tag intersystemscommunity/irispy:latest ``.


-:


docker run -d \
  -p 52773:52773 \
  -v /<HOST-DIR-WITH-iris.key>/:/mount \
  --name irispy \
  intersystemscommunity/irispy:latest \
  --key /mount/iris.key

, InterSystems IRIS Community Edition, .



  • isc.py.test.Process . , . WorkingDir .
  • : docker exec -it irispy sh.
  • SuperUser/SYS.
  • , : docker stop irispy && docker rm --force irispy.


, Python Gateway, , . InterSystems IRIS:


set sc = ##class(isc.py.Callout).Setup() 
set sc = ##class(isc.py.Main).SimpleString("x='HELLO'", "x", , .var).
write var

HELLO — Python- x. sc var , Readme — Troubleshooting section.


API


Python Gateway , , . !
Python isc.py.Main. ( %Status):




Python-.


SimpleString


SimpleString — . 4 :


  • code — . : $c(10).
  • returnVariable — .
  • serialization — returnVariable. 0 — string ( ), 1 — repr.
  • result — ByRef , returnVariable.

:


set sc = ##class(isc.py.Main).SimpleString("x='HELLO'", "x", , .var).

Python x Hello Python x ObjectScript var.


ExecuteCode


ExecuteCode SimpleString.
InterSystems IRIS 3 641 144 , , .
:


  • code — Python .
  • variable — () code Python .

:


set sc = ##class(isc.py.Main).ExecuteCode("2*3", "y").

2 3 Python y.



Python .


Python -> InterSystems IRIS


4 Python InterSystems IRIS, , :


  • String .
  • Repr .
  • JSON InterSystems IRIS.
  • Pickle .

Python .


  • GetVariable(variable, serialization, .stream, useString) — serialization variable stream. useString 1 , .
  • GetVariableJson(variable, .stream, useString) — JSON .
  • GetVariablePickle(variable, .stream, useString, useDill) - Pickle ( Dill) .

y.


set sc = ##class(isc.py.Main).GetVariable("y", , .val, 1)
write val
>6

InterSystems IRIS -> Python


InterSystems IRIS Python.


  • ExecuteQuery(query, variable, type, namespace) — (pandas dataframe list) sql Python variable. isc.py namespace — .
  • ExecuteGlobal(global, variable, type, start, end, mask, labels, namespace) — global start end Python type: list, pandas dataframe. mask and labels Data Transfer docs.
  • ExecuteClass(class, variable, type, start, end, properties, namespace) — class id start end Python type: list, pandas dataframe. properties — ( — ) , . * ?. — * ( ). %%CLASSNAME .
  • ExecuteTable(table, variable, type, start, end, properties, namespace) — table id start end Python.

ExecuteQuery — ( SQL- Python). , ExecuteGlobal ExecuteClass ExecuteTable . ( 3-5 ODBC- 20 ExecuteQuery). Data Transfer docs.
. isc.py .


ExecuteQuery


ExecuteQuery(request, variable, type, namespace) — SQL- Python. . , ExecuteGlobal .


:


  • query — sql .
  • variable — Python .
  • type — list Pandas dataframe.
  • namespace — .

ExecuteGlobal


ExecuteGlobal(global, variable, type, start, end, mask, labelels, namespace) — Python.


:


  • global — ^
  • variable — Python .
  • type — list Pandas dataframe.
  • start — . %Integer.
  • end — . %Integer.
  • mask — . , ( ). :
    • + .
    • - .
    • b — (0 — False, — True).
    • d — ( $horolog, Windows 1970, Linux 1900).
    • t — ($horolog, ).
    • m — ( YEAR-MONTH-DAY HOUR:MINUTE:SECOND).
  • labels — %List . — .
  • namespace — .

ExecuteClass


ExecuteGlobal. ExecuteGlobal .


ExecuteClass(class, variable, type, start, end, properties, namespace) — Python.


:


  • class —
  • variable — Python .
  • type — list Pandas dataframe.
  • start — Id.
  • end — Id
  • properties — ( — ) , . * ?. — * ( ). %%CLASSNAME .
  • namespace — .

%Date, %Time, %Boolean %TimeStamp — Python.


ExecuteTable


ExecuteClass. ExecuteClass. :


ExecuteTable(table, variable, type, start, end, properties, namespace) — Python.


:


  • table — .
    ExecuteClass.


  • ExecuteGlobal, ExecuteClass ExecuteTable .
  • ExecuteGlobal 20 ExecuteQuery ( >0.01 ).
  • ExecuteGlobal, ExecuteClass ExecuteTable : ^global(key) = $lb(prop1, prop2, ..., propN) key — .
  • ExecuteGlobal, ExecuteClass ExecuteTable %Date mktime (windows: 1970-01-01, linux 1900-01-01, mac). %TimeStamp pandas dataframe .. .
  • ExecuteGlobal, ExecuteClass ExecuteTable (, ) .


isc.py.test.Person , :


set global = "isc.py.test.PersonD"
set class = "isc.py.test.Person"
set table = "isc_py_test.Person"
set query = "SELECT * FROM isc_py_test.Person"

//  
set variable = "df"
set type = "dataframe"
set start = 1
set end = $g(^isc.py.test.PersonD, start)

//  0: ExecuteGlobal  
set sc = ##class(isc.py.Main).ExecuteGlobal(global, variable _ 0, type)

//  1: ExecuteGlobal      
//       
// globalKey -   
set labels = $lb("globalKey", "Name", "DOB", "TS", "RandomTime", "AgeYears", "AgeDecimal", "AgeDouble", "Bool")

// mask   1    labels   "globalKey" -  
//  %%CLASSNAME
set mask = "-+dmt+++b"

set sc = ##class(isc.py.Main).ExecuteGlobal(global, variable _ 1, type, start, end, mask, labels)

//  2: ExecuteClass
set sc = ##class(isc.py.Main).ExecuteClass(class, variable _ 2, type, start, end)

//  3: ExecuteTable
set sc = ##class(isc.py.Main).ExecuteTable(table, variable _ 3, type, start, end)

//  4: ExecuteTable
set sc = ##class(isc.py.Main).ExecuteQuery(query, variable _ 4, type)

do ##class(isc.py.test.Person).Test() .



  • GetVariableInfo(variable, serialization, .defined, .type, .length) — : , .
  • GetVariableDefined(variable, .defined) — .
  • GetVariableType(variable, .type) — .
  • GetStatus() — Python.
  • GetModuleInfo(module, .imported, .alias) — .
  • GetFunctionInfo(function, .defined, .type, .docs, .signature, .arguments) — .


Python Gateway , . Python — isc.py.ens.Operation. :


  • Python
  • / Python
  • Python

, Pyhton isc.py.Main. isc.py.ens.Operation Python InterSystems IRIS. :


  • isc.py.msg.ExecutionRequest Python . isc.py.msg.ExecutionResponse .
  • isc.py.msg.StreamExecutionRequest Python . isc.py.msg.StreamExecutionResponse . isc.py.msg.ExecutionRequest, .
  • isc.py.msg.QueryRequest SQL . Ens.Response.
  • isc.py.msg.GlobalRequest/isc.py.msg.ClassRequest/isc.py.msg.TableRequest //. Ens.Response.
  • isc.py.msg.SaveRequest Python . Ens.StringResponse .

  • isc.py.msg.RestoreRequest Python .


    , isc.py.ens.Operation :


    • Initializer — , isc.py.init.Abstract. , , .. .
    • PythonLib — ( Linux) , libpython3.6m.so Python.


-


, -:


  • isc.py.ens.ProcessUtils .
  • isc.py.util.BPEmulator - Python. - ( Python) .


-, isc.py.ens.ProcessUtils, GetAnnotation(name) . , InterSystems IRIS Python. :


  • ${class:method:arg1:...:argN} —
  • #{expr} — ObjectScript.

- isc.py.test.Process, , Correlation Matrix: Graph: f.savefig(r'#{process.WorkDirectory}SHOWCASE${%PopulateUtils:Integer:1:100}.png'). :


  • #{process.WorkDirectory} WorkDirectory process, isc.py.test.Process .. -.
  • ${%PopulateUtils:Integer:1:100} Integer %PopulateUtils, 1 100, 1...100.

-


- Python Gateway. :


  1. : pip install pandas matplotlib seaborn.
  2. InterSystems IRIS : do ##class(isc.py.test.CannibalizationData).Import() .
  3. isc.py.test.Production.
  4. Ens.Request isc.py.test.Process.

, . isc.py.test.Process BPL:




— Python :



isc.py.msg.ExecutionRequest, :


  • Code — Python .
  • SeparateLines — . $c(10) (\n) . , , def . 0.
  • Variables — , .
  • Serialization — , . : Str, Repr, JSON, Pickle Dill, Str.

, Code, . process.GetAnnotation("Import pandas"), . , import pandas as pd Python. GetAnnotation Python , . Code .



isc.py.msg.ExecutionRequest — Correlation Matrix: Tabular:



Python corrmat InterSystems IRIS JSON, :


  • Variables: "corrmat"
  • Serialization: "JSON"

Visual Trace:



, : callresponse.Variables.GetAt("corrmat").



InterSystems IRIS Python, isc.py.msg.DataRequest, :


  • Variable — Python .
  • Type — : dataframe (pandas dataframe) list.
  • Namespace — . isc.py . .

4 :


  • isc.py.msg.QueryRequest — Query SQL .
  • isc.py.msg.ClassRequest — Class .
  • isc.py.msg.TableRequest — Table .
  • isc.py.msg.GlobalRequest — Global .

RAW, isc.py.msg.QueryRequest .



/ Python


, Python InterSystems IRIS, , isc.py.msg.SaveRequest :


  • Mask — , . * ?. : "Data*, Figure?". *.
  • MaxLength — . , . 0, . $$$MaxStringLength.
  • Name — ().
  • Description — ().

Ens.StringResponse Id . Save Context.


isc.py.msg.RestoreRequest InterSystems IRIS Python:


  • ContextId — .
  • Clear — .

Jupyter Notebook


Jupyter Notebook — - , , , . Python Gateway BPL- Jupyter Notebook. , executor Python 3.


, Python . - PythonGateway Jupyter Notebook. :


  • -
  • -

-. :







  1. InterSystems IRIS 2019.2+.
  2. PythonGateway v0.8+ ( isc.py.util.Jupyter, isc.py.util.JupyterCheckpoints isc.py.ens.ProcessUtils).
  3. ObjectScript .
  4. do ##class(isc.py.util.Jupyter).Install() .

.



MLToolkit — , , -. Python Gateway MLToolkit Python , Python ( Data Scientists), , AI/ML- InterSystems IRIS.




MLToolkit


MLToolkit — GitHub-, GitHub- InterSystems. , , MLToolkit, Python Gateway. ( ) , , . ML Toolkit, , : MLToolkit@intersystems.com :


  • GitHub
  • ( )
  • ( , "", "").

, InterSystems IRIS , , . ; AI/ML – MLToolkit@intersystems.com.

Source: https://habr.com/ru/post/undefined/

More articles:


All Articles