Интеграция драгоценного камня поиска Sunspot solr в gitlab ci

Я разрабатываю приложение для рельсов и использую драгоценный камень солнечных пятен для поиска.

gem 'sunspot_solr' 

Я настроил его на своем компьютере, и он отлично работает. Также я интегрирую свое приложение в gitlab ci. До того, как я настроил поиск, он был успешно построен. И теперь я добавляю службу solr в файл gitlab-ci.yml, использую запуск и переиндексацию.

.gitlab-ci.yml

services:
  - postgres
  - solr
...

rake_test:
  script:
    - bundle exec sunspot-solr start -p 8983
    - bundle exec rake db:create
    - bundle exec rake db:test:prepare
    - bundle exec rake db:migrate
    - bundle exec rake sunspot:solr:reindex

config/sunspot.yml

production:
  solr:
    hostname: <%= ENV['SOLR_HOST'] || 'localhost' %>
    port: <%= ENV['SOLR_PORT'] || '8983' %>
    log_level: WARNING
    path: <%= ENV['SOLR_PATH'] || '/var/solr' %>
    # read_timeout: 2
    # open_timeout: 0.5

development:
  solr:
    hostname: localhost
    port: 8983
    log_level: INFO

test:
  solr:
    hostname: localhost
    port: 8983
    log_level: WARNING

Лог из сборки gitlab:

bundle exec sunspot-solr start -p 8983
bundle exec rake sunspot:solr:reindex
Skipping progress bar: for progress reporting, add gem 'progress_bar' to your Gemfile
rake aborted!
RSolr::Error::ConnectionRefused: Connection refused - {:data=>"<?xml version=\"1.0\" encoding=\"UTF-8\"?><delete><query>type:Product</query></delete>", :headers=>{"Content-Type"=>"text/xml"}, :method=>:post, :params=>{:wt=>:ruby}, :query=>"wt=ruby", :path=>"update", :uri=>#<URI::HTTP http://localhost:8983/solr/update?wt=ruby>, :open_timeout=>nil, :read_timeout=>nil, :retry_503=>nil, :retry_after_limit=>nil}
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/connection.rb:19:in `rescue in execute'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/connection.rb:14:in `execute'
/usr/local/bundle/gems/rack-mini-profiler-0.10.1/lib/patches/db/rsolr.rb:4:in `execute_with_profiling'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/client.rb:181:in `execute'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/client.rb:175:in `send_and_receive'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/solr_instrumentation.rb:16:in `block in send_and_receive_with_as_instrumentation'
/usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/notifications.rb:164:in `block in instrument'
/usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/notifications.rb:164:in `instrument'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/solr_instrumentation.rb:15:in `send_and_receive_with_as_instrumentation'
(eval):2:in `post'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/client.rb:84:in `update'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/client.rb:148:in `delete_by_query'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/indexer.rb:68:in `remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session.rb:197:in `block in remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session.rb:197:in `each'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session.rb:197:in `remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session_proxy/retry_5xx_session_proxy.rb:17:in `method_missing'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot.rb:502:in `remove_all'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/method_tracer.rb:364:in `block in remove_all_with_trace_SolrClient_Sunspot_delete'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/method_tracer_helpers.rb:82:in `trace_execution_scoped'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/method_tracer.rb:362:in `remove_all_with_trace_SolrClient_Sunspot_delete'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/searchable.rb:190:in `solr_remove_all_from_index'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/searchable.rb:209:in `solr_reindex'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/tasks.rb:60:in `block (4 levels) in <top (required)>'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/class_set.rb:16:in `each'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/class_set.rb:16:in `each'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/tasks.rb:59:in `block (3 levels) in <top (required)>'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/tasks.rb:71:in `with_session'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/tasks.rb:19:in `block (2 levels) in <top (required)>'
/usr/local/bundle/gems/rake-11.2.2/exe/rake:27:in `<top (required)>'
/usr/local/bundle/bin/bundle:22:in `load'
/usr/local/bundle/bin/bundle:22:in `<main>'
Errno::ECONNREFUSED: Failed to open TCP connection to localhost:8983 (Connection refused - connect(2) for "localhost" port 8983)
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/instrumentation/net.rb:27:in `block (2 levels) in request_with_newrelic_trace'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent.rb:428:in `disable_all_tracing'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/instrumentation/net.rb:26:in `block in request_with_newrelic_trace'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/cross_app_tracing.rb:48:in `tl_trace_http_request'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/instrumentation/net.rb:23:in `request_with_newrelic_trace'
/usr/local/bundle/gems/rack-mini-profiler-0.10.1/lib/patches/net_patches.rb:7:in `block in request_with_mini_profiler'
/usr/local/bundle/gems/rack-mini-profiler-0.10.1/lib/mini_profiler/profiling_methods.rb:37:in `step'
/usr/local/bundle/gems/rack-mini-profiler-0.10.1/lib/patches/net_patches.rb:6:in `request_with_mini_profiler'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/connection.rb:15:in `execute'
/usr/local/bundle/gems/rack-mini-profiler-0.10.1/lib/patches/db/rsolr.rb:4:in `execute_with_profiling'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/client.rb:181:in `execute'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/client.rb:175:in `send_and_receive'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/solr_instrumentation.rb:16:in `block in send_and_receive_with_as_instrumentation'
/usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/notifications.rb:164:in `block in instrument'
/usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/notifications.rb:164:in `instrument'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/solr_instrumentation.rb:15:in `send_and_receive_with_as_instrumentation'
(eval):2:in `post'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/client.rb:84:in `update'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/client.rb:148:in `delete_by_query'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/indexer.rb:68:in `remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session.rb:197:in `block in remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session.rb:197:in `each'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session.rb:197:in `remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session_proxy/retry_5xx_session_proxy.rb:17:in `method_missing'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot.rb:502:in `remove_all'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/method_tracer.rb:364:in `block in remove_all_with_trace_SolrClient_Sunspot_delete'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/method_tracer_helpers.rb:82:in `trace_execution_scoped'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/method_tracer.rb:362:in `remove_all_with_trace_SolrClient_Sunspot_delete'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/searchable.rb:190:in `solr_remove_all_from_index'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/searchable.rb:209:in `solr_reindex'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/tasks.rb:60:in `block (4 levels) in <top (required)>'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/class_set.rb:16:in `each'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/class_set.rb:16:in `each'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/tasks.rb:59:in `block (3 levels) in <top (required)>'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/tasks.rb:71:in `with_session'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/tasks.rb:19:in `block (2 levels) in <top (required)>'
/usr/local/bundle/gems/rake-11.2.2/exe/rake:27:in `<top (required)>'
/usr/local/bundle/bin/bundle:22:in `load'
/usr/local/bundle/bin/bundle:22:in `<main>'
Errno::ECONNREFUSED: Connection refused - connect(2) for "localhost" port 8983
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/instrumentation/net.rb:27:in `block (2 levels) in request_with_newrelic_trace'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent.rb:428:in `disable_all_tracing'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/instrumentation/net.rb:26:in `block in request_with_newrelic_trace'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/cross_app_tracing.rb:48:in `tl_trace_http_request'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/instrumentation/net.rb:23:in `request_with_newrelic_trace'
/usr/local/bundle/gems/rack-mini-profiler-0.10.1/lib/patches/net_patches.rb:7:in `block in request_with_mini_profiler'
/usr/local/bundle/gems/rack-mini-profiler-0.10.1/lib/mini_profiler/profiling_methods.rb:37:in `step'
/usr/local/bundle/gems/rack-mini-profiler-0.10.1/lib/patches/net_patches.rb:6:in `request_with_mini_profiler'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/connection.rb:15:in `execute'
/usr/local/bundle/gems/rack-mini-profiler-0.10.1/lib/patches/db/rsolr.rb:4:in `execute_with_profiling'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/client.rb:181:in `execute'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/client.rb:175:in `send_and_receive'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/solr_instrumentation.rb:16:in `block in send_and_receive_with_as_instrumentation'
/usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/notifications.rb:164:in `block in instrument'
/usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/usr/local/bundle/gems/activesupport-4.2.7/lib/active_support/notifications.rb:164:in `instrument'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/solr_instrumentation.rb:15:in `send_and_receive_with_as_instrumentation'
(eval):2:in `post'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/client.rb:84:in `update'
/usr/local/bundle/gems/rsolr-1.1.1/lib/rsolr/client.rb:148:in `delete_by_query'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/indexer.rb:68:in `remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session.rb:197:in `block in remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session.rb:197:in `each'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session.rb:197:in `remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session_proxy/retry_5xx_session_proxy.rb:17:in `method_missing'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `remove_all'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot.rb:502:in `remove_all'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/method_tracer.rb:364:in `block in remove_all_with_trace_SolrClient_Sunspot_delete'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/method_tracer_helpers.rb:82:in `trace_execution_scoped'
/usr/local/bundle/gems/newrelic_rpm-3.16.0.318/lib/new_relic/agent/method_tracer.rb:362:in `remove_all_with_trace_SolrClient_Sunspot_delete'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/searchable.rb:190:in `solr_remove_all_from_index'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/searchable.rb:209:in `solr_reindex'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/tasks.rb:60:in `block (4 levels) in <top (required)>'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/class_set.rb:16:in `each'
/usr/local/bundle/gems/sunspot-2.2.5/lib/sunspot/class_set.rb:16:in `each'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/tasks.rb:59:in `block (3 levels) in <top (required)>'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/tasks.rb:71:in `with_session'
/usr/local/bundle/gems/sunspot_rails-2.2.5/lib/sunspot/rails/tasks.rb:19:in `block (2 levels) in <top (required)>'
/usr/local/bundle/gems/rake-11.2.2/exe/rake:27:in `<top (required)>'
/usr/local/bundle/bin/bundle:22:in `load'
/usr/local/bundle/bin/bundle:22:in `<main>'
Tasks: TOP => sunspot:solr:reindex => sunspot:reindex
(See full trace by running task with --trace)
make: *** [solr_start] Error 1
Makefile:30: recipe for target 'solr_start' failed
ERROR: Build failed: exit code 1

Как видите, начинается хорошо. Но на этапе индексации он не мог подключиться.

Я также пробовал команду ps aux

после запуска Solr:

ps aux | grep solr

root     15048  0.0  0.5  95024 22168 ?        R    15:26   0:00 /usr/local/bundle/bin/sunspot-solr                                              
root     15051  0.0  0.0  20128  2816 ?        Ss   15:26   0:00 bash ./solr start -f -p 8983 -s /usr/local/bundle/gems/sunspot_solr-2.2.5/solr/solr
root     15056  0.0  0.0  11132  1000 ?        S    15:26   0:00 grep solr

и после переноса БД:

ps aux | grep solr

root     15066  0.0  0.0  11132   988 ?        S    15:27   0:00 grep solr

person Павел Леонюк    schedule 25.08.2016    source источник


Ответы (1)


В моем случае я расширил StubSessionProxy https://github.com/jaigouk/stub_solr.

вот мой файл рейка https://github.com/jaigouk/stub_solr/blob/master/Rakefile#L21

person Jaigouk    schedule 14.09.2016
comment
Пожалуйста, не публикуйте только ссылки. Поместите соответствующие фрагменты кода прямо в свой пост (иначе он станет неактуальным, если ссылка не работает) - person YakovL; 14.09.2016