Sr. Software Engineer - Professional Archive Search & AI at smarsh

As a Senior Software Engineer on the Professional Archive Search team, you’ll help design, build, and evolve advanced search and data systems that power our platform. Your work will focus on blending traditional search with modern AI-driven approaches, including vector search, knowledge graphs, and retrieval-augmented generation (RAG). You’ll play a key role in enabling Smarsh clients to meet their compliance needs by delivering scalable, reliable, and high-performance search capabilities. This includes working with high-dimensional data, real-time pipelines, and hybrid search systems. You’ll collaborate with cross-functional partners across Product Management, Engineering, and Site Reliability to solve complex challenges and shape the future of search within our platform. We’re looking for someone who is thoughtful, curious, and excited to work at the intersection of search and AI. How will you contribute? • Support and empower your team by contributing to a collaborative, inclusive, and respectful work environment • Partner with engineers and stakeholders to design and deliver scalable search and data solutions • Help bridge traditional lexical search and modern semantic search approaches • Design and implement hybrid search strategies combining keyword-based and vector-based retrieval • Work with vector databases to manage and query high-dimensional embeddings • Design and optimize retrieval-augmented generation (RAG) pipelines, including handling long documents and retrieval quality • Contribute to knowledge graph–driven approaches to enhance search relevance and data relationships • Build and maintain high-throughput streaming data pipelines (e.g., Kafka) for real-time ingestion and indexing • Contribute to architectural decisions that improve system scalability, reliability, and observability • Participate in a shared on-call rotation to support service reliability and incident response with a focus on learning and prevention • Collaborate with Product and Engineering to define technical requirements, timelines, and deliverables • Apply modern engineering practices, including Agile methodologies, CI/CD pipelines, and DevOps principles • Review code, identify areas for improvement, and help reduce technical debt • Troubleshoot and resolve production issues to maintain high availability and performance • Stay current with emerging technologies in search, AI, and data systems, and evaluate their impact • Deploy and manage applications in Kubernetes environments • Monitor application health and performance using tools such as Splunk, Datadog, and Grafana What will you bring? We’re looking for someone who enjoys working collaboratively, values continuous learning, and communicates openly and respectfully. You support others, share knowledge, and contribute to an inclusive and positive team culture. • Experience building and operating software in a modern private cloud-based environment • Strong background in distributed systems and search or data-intensive applications • Experience with search technologies such as Lucene, Elasticsearch, or Solr, including how indexing and query systems work • Hands-on experience with vector similarity search and vector databases (e.g., Qdrant, Milvus, Vespa, or similar) • Familiarity with retrieval-augmented generation (RAG) concepts and architectures • Understanding of knowledge graphs or graph-based data modeling concepts • Experience designing scalable data pipelines and working with streaming technologies such as Kafka • A collaborative mindset, with openness to feedback and different perspectives • Understanding of modern software development practices and Agile methodologies • Ability to communicate technical concepts, progress, and trade-offs clearly • Comfort working in evolving environments where requirements may change over time • A proactive and thoughtful approach to solving complex problems Preferred Qualifications • Around 6+ years of experience in software engineering, with experience in search or data systems • Proficiency in Java or a similar backend programming language • Experience deploying and managing applications in Kubernetes • Familiarity with relational databases and query optimization (e.g., MS SQL or similar) • Experience working with Linux-based systems • Exposure to messaging systems such as Kafka or AMQ • Experience contributing to or designing AI/ML-powered systems in production environments • Participation in open-source projects or technical communities • The ProArchive Application Development team embraces an AI-first mindset, continuously elevating developer productivity through tools like GitHub Copilot, Windsurf, and Claude Code. We’re seeking developers who are passionate about advancing their AI-assisted development skills and leveraging them to deliver innovative, high-impact solutions for Smarsh customers. What do we offer? • Healthcare insurance: We provide medical, dental, and vision insurance, and a flexible spending account that allows you to set aside pre-tax dollars to pay for eligible out-of-pocket expenses. • Stock options. • Personal time off: A healthy work-life balance is critical to your success at the office. Smarsh offers a “take-what-you-need” time off policy as well as flexible work arrangements. • 401K Match: Smarsh provides a 4% 401K match for which employees are fully vested on day one. • Sabbatical: The Smarsh sabbatical programme provides a time to recharge, study or simply do something you are passionate about away from the workplace. Employees are eligible after six years of service. • Recognition: We’re big on kudos for a job well done. Our employee-recognition programme enables co-workers to nominate their peers who best embody our core values for recognition. Don't meet every requirement? Apply anyway! We value diverse candidates and encourage applications, even if you don't perfectly match the job description. Studies have shown that some strong candidates may self-select out of the interview process prematurely, at Smarsh we encourage an inclusive, high-performing environment.   Smarsh is an equal opportunity and affirmative action employer. Qualified applicants will receive consideration without regard to their race, colour, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran. Smarsh invites all qualified interested applicants to apply for career opportunities. Reasonable accommodations may be made to enable individuals with disabilities to perform essential functions. Including frequency of functions