Optimizamos la memoria del servicio Rails (caso real)

Para Ruby y Ruby on Rails Junior +, Desarrolladores intermedios

A menudo escribimos código sin profundizar en cuántos recursos se necesitarán para ejecutarlo. Y podría estar bien. Pero, en primer lugar, cada desarrollador normal está tratando de mejorar y cambiando sus habilidades. En segundo lugar, esto puede convertirse fácilmente en un error real con raíces absolutamente no obvias, y la pregunta aún debe abordarse.

Bicho misterioso

, CRM Kubernetes. Excel , . Sidekiq - , ( exception-, ). ‘in_progress’, Sidekiq , .

, , , , , . , CRM, , .

""

"" — . , - , . benchmark-memory, :

def benchmark_memory(&block)
  Benchmark.memory do |x|
    x.report('') do
      yield
      GC.start(full_mark: true, immediate_sweep: true) #   Garbage Collector',     
    end
  end
end

Garbage Collector . «» . GC Ruby , . , , . , , 100 GC:

 100    

50.966M memsize (     1.445M retained)
850.405k objects (    24.514k retained)
50.000  strings (    50.000  retained)

  100    

80.554M memsize (   920.363k retained)
 1.303M objects (    17.565k retained)
50.000  strings (    50.000  retained)

, Garbage Collector’, :

benchmark_memory { puts 1 }

Calculating -------------------------------------
1
                       464.000  memsize (     0.000  retained)
                         2.000  objects (     0.000  retained)
                         0.000  strings (     0.000  retained)

, 464 .

:

 100 :

1.482M memsize (     1.480M retained)
24.814k objects (    24.793k retained)
50.000  strings (    50.000  retained)

 1000 :

12.811M memsize (    12.809M retained)
226.587k objects (   226.556k retained)
50.000  strings (    50.000  retained)

100 , 1000 — . — , . , , , ( ). , , .

, , , . Garbage Collector’ 100 — 50 ( ), 500 1000. 38_000 — "" 18,5 ! (38 * 500 = 19_000 =~ 18,5 ).

?

1:

, — , xlsx csv. «», , . , «» , 40% :

 100   xlsx:

1.482M memsize (     1.480M retained)
24.814k objects (    24.793k retained)
50.000  strings (    50.000  retained)

 100   csv:

852.195k memsize (   849.968k retained)
12.034k objects (    12.007k retained)
50.000  strings (    50.000  retained)

2:

:

, , . ReportWorker’ . . , :

  def export_collection(objects_ids)
    Application.where(id: objects_ids.uniq).order(created_at: :desc)
  end

, , ? ActiveRecord find_each:

 def to_csv_export(objects_ids)
    CSV.open(file_path, 'w') do |csv|
      csv << export_headers #    

      export_collection(objects_ids).find_each do |application| #     ,     
        begin
          csv << export_row(application.decorate) #     
        rescue StandardError => e
          errors << "#{application.id}: #{e}"
        end
      end
    end
  end

, :

 100 :

166.688k memsize (   165.445k retained)
1.729k objects (     1.715k retained)
50.000  strings (    50.000  retained)

 1000 :

167.510k memsize (   166.227k retained)
1.734k objects (     1.719k retained)
50.000  strings (    50.000  retained)

, , , , .

:

, , — , 18,5 , 200 . , , , , . , , . . , , .

: 170 ,